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Executive  Summary 


Background 

The  objective  of  this  task,  sponsored  by  the  Deputy  Assistant  Secretary  of  Defense 
for  Counternarcotics  and  Global  Threats  (CN&GT),  was  to  improve  the  effectiveness  of 
the  Counternarcotics  Police  of  Afghanistan  (CNPA).  In  addition,  the  Institute  for  Defense 
Analyses  (IDA)  supported  the  United  States  Central  Command’s  (CENTCOM’s) 
Counternarcotics  Program  in  the  Interagency  Action  Group  (IAG),  and  received 
additional  requirements  from  CENTCOM.  CENTCOM  sought  data  to  drive  resourcing 
decisions,  i.e.,  to  identify  functional  areas  where  the  United  States  Government  (USG) 
should  concentrate  CN  resources.  Finally,  multiple  countemarcotics  program  goals  would 
be  achieved  through  the  Defense  Intelligence  Agency’s  (DIA’s)  effort  to  develop  data 
collection  and  validation  procedures  for  Southwest  Asia  (SWA)  that  are  consistent  with 
the  Western  Hemisphere’s  Consolidated  Counterdrug  Database  (CCDB)  process. 
Accordingly,  IDA’s  efforts  were  directed  toward  supporting  this  DIA  effort  and  creating 
a  CCDB-East. 

To  support  CCDB-East  development,  IDA  analyzed  drug  seizure  event  data  from 
the  United  Nations  Office  of  Drugs  and  Crime  (UNODC)  and  data  extracts  used  by  DIA 
to  assess  their  quality  and  utility  for  this  purpose.  IDA  studied  and  observed  the  existing 
process  for  collecting,  validating,  and  analyzing  cocaine  data  in  the  Western  Hemisphere 
and  how  it  could  be  applied  in  SWA.  Finally,  IDA  devised  techniques  that  would  allow 
analysts  to  derive  useful  infonnation  from  limited  data  sets.  This  document  presents  these 
techniques  and  shows  how  they  can  be  employed  in  SWA  to  assist  the  USG  in  measuring 
the  success  of  CENTCOM’s  CN  programs. 

Findings  and  Recommendations 

The  following  findings  and  recommendations  highlight  how  understanding  patterns 
in  drug  data  enables  better  analysis. 

1.  Finding:  Raw,  not  averaged,  data  are  necessary  for  more  accurate  analyses. 

Raw  data,  i.e.,  individual  values  collected  from  the  field,  are  necessary  to  accurately 
calculate  the  sample  means,  data  distributions,  and  medians  used  for  statistical  analyses. 
Depending  on  the  sample  size,  there  can  be  significant  divergence  between  means  and 
median  values.  For  certain  applications,  the  sample  mean  will  be  much  higher  than  the 
sample  median.  Moreover,  the  sample  mean  is  highly  erratic  while  medians  can  better 
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reveal  true  trends.  Thus  interpretations  based  on  a  sample  mean  may  lead  to  erroneous 
conclusions.  Moreover,  raw  data  are  better  for  making  comparisons  to  power  law 
distributions, 1  providing  some  sense  of  statistical  variation,  and  offering  more  support  for 
trend  analysis. 

Recommendation:  Entries  in  the  CCDB-East  should  be  in  the  form  of  raw  event 
data,  not  averaged  estimates  (i.e.,  sample  mean  and  sample  median). 

2.  Finding:  It  is  essential  to  maintain  the  integrity  of  separate  data  sets. 

Combining  two  or  more  data  sets  obscures  the  time-dependent  infonnation 
contained  within  the  data  and  renders  it  unusable  for  validation  purposes. 

Recommendation:  Analyses  of  the  data  contained  in  the  CCDB-East  should 
maintain  the  integrity  of  separate  data  sets. 

3.  Finding:  Systematized  data  collection  methods  yield  more  reliable  information. 

Coordinated  data  collection  and  information  sharing  efforts  across  civilian  and 
military  government  agencies  that  have  some  role  to  play  in  the  countemarcotics  mission 
yield  a  more  robust  data  set  that  can  be  used  to  infonn  policy  and  extract  other  useful 
information. 

Recommendation:  Formal  coordination  of  the  data  collection  processes  should  be 
implemented  among  the  interagency  and  multinational  partners. 

4.  Finding:  Event  data  should  be  categorized  by  confidence  levels  before  analysis. 

Confidence  levels  are  assigned  by  assessing  the  reliability  of  each  event  based  on 
multiple  sources  of  corroborating  data.  This  allows  analysts  to  use  data  in  different  ways, 
including  as  a  validation  tool  for  other  data  sources. 

Recommendation:  The  CCDB-East  should  have  a  system  for  assigning  and  vetting 
event  data  confidence  levels. 

5.  Finding:  Event  data  may  be  validated  using  independent  variables  that  are 
known  to  indirectly  corroborate  event  data. 

Comparing  separate  but  related  independent  variables,  such  as  price  or  user  rates, 
reveal  similarities  as  well  as  inconsistencies  that  reflect  the  quality  of  the  data  and 
potential  anomalies  that  require  further  examination.  For  example,  changes  in  seizure 
rates  may  also  be  reflected  in  changes  in  prices  or  consumption. 


A  power  law  is  a  distribution  where  rates  change  as  the  size  of  events  and  risk  levels  grows.  This 
concept  is  examined  in  detail  in  this  document. 
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Recommendation:  Data  in  the  CCDB-East  should  be  compared  with  independent 
data  sets  that  provide  some  validation  potential. 

6.  Finding:  Event  data  may  be  validated  using  time  series  plots. 

Comparing  different  variables  over  time  allows  analysts  to  identify  reinforcing 
trends  and  anomalies,  and  compels  analysts  to  account  for  differences  in  interpretations 
of  causal  factors. 

Recommendation:  Data  in  the  CCDB-East  should  be  compared  with  independent 
data  sets  that  provide  some  validation  potential. 

7.  Finding:  Event  data  may  be  validated  by  comparing  it  with  the  underlying 
functional  distributions  (power  laws). 

Because  many  sets  of  drug  data,  especially  observed  counts  for  specific  sizes  of 
events,  follow  similar  functional  distributions,  data  sets  can  be  compared  with  each  other 
for  consistency.  When  data  sets  diverge  from  the  expected  distribution,  one  can 
reasonably  infer  that  the  data  set  may  be  incomplete,  contaminated,  or  suffer  from  some 
other  fundamental  shortcoming. 

Recommendation:  Administrators  of  the  CCDB-East  should  ensure  the  thorough 
technical  validation  of  all  event-level  data  to  maximize  the  utility  of  limited  data 
sets. 

Recommendation:  CCDB-East  administrators  and  data  collectors  should  agree  on 
minimal  reporting  thresholds  that  accurately  sample  the  universe  of  drug  trafficking 
events. 

8.  Finding:  CENTCOM  should  extract  surrogate  data  where  incomplete  data  sets 
exist. 

New  analytic  methods  can  be  used  to  extract  additional  data  from  existing  data  sets 
providing  the  potential  for  insight  into  otherwise  unknown  aspects  of  the  drug  trade.  For 
these  methods  to  work,  small  trafficking  events  down  to  one  kilogram  (kg)  must  be 
collected. 

Recommendation:  CENTCOM  should  extract  surrogate  data  where  incomplete 
data  sets  exist. 

9.  Finding:  Better  event  data  can  support  CENTCOM’s  programs  by  mitigating 
some  of  the  challenges  associated  with  data  limitations. 

Various  interagency  stakeholders  have  identified  specific  data  gaps  where  additional 
information  could  be  extremely  helpful,  such  as  the  location  of  stockpiles,  processing 
labs,  routes,  and  the  supply  of  precursor  chemicals.  These  data,  however,  have  limited 
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availability.  Moreover,  USG  data  collection  efforts  are  neither  coordinated  nor 
systematic,  resulting  in  incomplete  data  sets  that  may  not  accurately  reflect  drug 
trafficking  in  SWA.  The  techniques  presented  in  this  document  offer  ways  to  validate 
existing  data,  fill  in  missing  data,  and  extract  supplemental  insights. 

Recommendation:  See  all  recommendations  associated  with  findings  1-8. 

10.  Finding:  CENTCOM  can  improve  its  knowledge  of  its  adversaries,  how  they 
resource  themselves,  and  how  they  operate  based  on  perceived  risk. 

This  finding  regarding  criminals’  tolerance  for  risk-taking  offers  opportunities  for 
the  USG  to  tailor  its  counternarcotics  activities.  By  creating  a  sufficient  level  of  risk  that 
deters  traffickers,  the  USG  can  make  progress  towards  achieving  its  objectives  and 
maximizing  its  returns  on  countemarcotics  investments. 

Recommendation:  See  all  recommendations  associated  with  findings  1-8. 

An  Analytic  Breakthrough 

In  executing  this  task,  IDA  made  a  breakthrough  in  its  broader  analysis  of  criminal 
systems.  Decades  of  research  on  the  cocaine  market,  this  research  on  opiates  in  SWA, 
and  research  on  the  behavior  of  insurgents  in  Iraq  have  suggested  that  irregular 
adversaries,  including  insurgents,  drug  traffickers,  and  other  criminal  enterprises,  all 
exhibit  similarly-structured  rates  (or  time  intervals  between  events)  of  risk-taking. 

While  many  will  take  small  risks  for  small  returns  because  the  perceived  risk  is  low, 
only  a  few  take  very  big  risks  for  large  returns.  For  example,  small  drug  seizures  of 
amounts  less  than  one  kilogram  are  far  more  frequent  than  seizures  of  a  ton.  Similarly, 
more  traffickers  sell  smaller  amounts  more  frequently  than  very  large  amounts.  Likewise, 
more  users  buy  small  amounts  more  often  than  users  who  buy  large  amounts.  In  other 
words,  the  rate  of  criminal  behavior  (i.e.,  the  interval  between  events)  is  faster  for  smaller 
risks  and  slower  for  larger  risks.  This  observation  is  central  to  the  principle  known  as 
equal  risks  for  equal  returns  and  to  the  associated  power  law  distributions  where  rates 
change  as  the  size  of  events  and  risk  levels  grows.  This  behavior  appears  to  be  a 
fundamental  signature  of  irregular  gangs,  insurgents,  traffickers,  and  any  other  small 
illicit  organization  avoiding  detection  by  a  state.  Because  these  rates  are  linked  to  each 
other,  analysts  may  be  able  to  derive  significant  amounts  of  infonnation  from  limited  data 
sets. 

The  research  presented  in  this  document  and  its  appendices  suggest  that  this  risk¬ 
taking  behavior  is  ubiquitous  and  applies  not  only  to  global  criminal  activities,  but  also  to 
insurgent  forces — many  of  which  are  funded  through  the  drug  trade."  Further  analysis  of 
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For  more  detail  on  analysis  of  combat  data,  see  Appendix  C,  “Analysis  of  Combat  Data.” 


the  limited  data  is  required  to  confirm  this  hypothesis,  but  assuming  it  is  correct,  it  could 
assist  the  U.S.  military  in  identifying  the  conditions  under  which  it  must  operate  (and  the 
risks  it  must  impose  upon  insurgents)  for  U.S.  forces  to  gain  the  advantage  in  irregular 
wars.  In  other  words,  it  could  improve  the  U.S.  military’s  knowledge  of  its  adversaries 
and  how  they  operate  based  on  perceived  risk. 

The  implication  for  the  USG  is  that  it  might  be  able  to  detennine — with  far  better 
accuracy — the  actual  scope  of  certain  aspects  of  the  drug  and  irregular  warfare  systems. 
With  more  accurate  data,  policy  makers  could  begin  to  pinpoint  with  better  precision 
when  certain  government  policies  and  interventions  begin  to  work. 
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1.  Introduction 


This  chapter  explains  the  objectives  and  methodology  the  Institute  for  Defenses 
Analyses  (IDA)  used  to  conduct  this  study,  describes  how  IDA  used  data  provided  by  the 
United  Nations  Office  on  Drugs  and  Crime  (UNODC)  and  provides  an  outline  of  the 
document’s  organization. 

A.  Objectives 

This  research  was  carried  out  in  support  of  the  Office  of  the  Deputy  Assistant 
Secretary  of  Defense  (DASD)  for  Countemarcotics  and  Global  Threats  (CN&GT).  The 
ultimate  objective  of  the  DASD-sponsored  effort  is  to  improve  the  effectiveness  of  the 
Counternarcotics  Police  of  Afghanistan  (CNPA). 

The  United  States  Central  Command  (CENTCOM)  was  also  pursuing  improvement 
in  data  quality  and  use  in  connection  with  the  CENTCOM  Counternarcotics  Program.  In 
particular,  CENTCOM  sought  better  data  to  drive  its  resourcing  decisions,  i.e.,  to 
detennine  the  functional  areas  to  concentrate  its  CN  resources  (e.g.,  border  enhancements 
or  intelligence-sharing  efforts,  interdiction  or  eradication,  training,  equipping,  or 
infrastructure  support,  etc.).  Moreover,  CENTCOM  was  focused  on  the  global  nature  of 
opiate  trafficking.  Although  more  than  90%  of  opiates  are  produced  in  CENTCOM’s  area 
of  responsibility  (AOR),  they  cross  several  Combatant  Command  (COCOM)  boundaries 
en  route  to  their  final  destination.  Thus,  the  methods  developed  to  answer  these  questions 
needed  to  be  holistic  so  that  the  principles  can  be  applied  by  any  COCOM. 

Simultaneously,  the  Defense  Intelligence  Agency  (DIA)  was  engaged  in  an  effort  to 
develop  an  opiate  module  within  the  Consolidated  Counterdrug  Database  (CCDB).  As 
part  of  this  effort,  DIA  is  preparing  for  data  collection  and  validation  procedures 
consistent  with  the  existing  CCDB  process  and  contained  within  that  database 
infrastructure. 1 

DIA  looks  for  event-level,  seizure  data  with  basic  descriptive  details  including  date, 
location,  drug  type,  amount,  destination,  description  of  where  it  was  seized,  the 
force/organization  that  claimed  the  seizure,  operation  type/name,  and  if  available,  a 


The  CCDB  is  a  repository  of  Western  Hemisphere  cocaine  event  data  from  drug  movements  and 
seizures.  It  captures  the  details  of  these  drug-related  events  and  supports  both  Interagency  Assessment 
of  Cocaine  Movement  (IACM)  and  Performance  Assessment  Review  (PAR)  data  requirements.  For 
more  information,  see  the  CCDB  User  Guide. 
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narrative  of  the  seizure.  DIA  is  seeking  this  level  of  detail  for  reports  from  Afghanistan, 
Turkey,  Iran,  Pakistan,  India,  Russia,  as  well  as  African  and  European  countries  as  a 
secondary  requirement.  Unlike  the  UNODC,  which  collects  regional  drug  data  in  a 
systematized  fashion,  early  DIA  collection  efforts  have  not  had  the  benefit  of  regional 
data  and  have  had  to  rely  largely  on  data  from  existing  databases  that  are  not  designed  to 
acquire  and  house  drug  data.  DIA  plans  to  use  these  reports  to  ensure  the  event  data 
within  the  opiate  module  of  the  CCDB  (hereafter  referred  to  as  “CCDB-East”)  is  not 
duplicative  and  to  de-conflict/combine  seizures  that  are  reported  or  claimed  by  more  than 

one  unit.  The  CCDB-East  calls  this  practice  “validation”  although  it  is  quite  different 
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from  the  technical  process  of  validation. 

Ultimately,  the  DIA  effort  to  create  a  “CCDB-East”  became  the  common  focus  for 
the  improvement  of  the  data  collection  and  analysis  capabilities  essential  to  the 
counternarcotics  efforts  of  the  United  States  and  the  CNPA.  IDA’s  efforts  were  directed 
toward  supporting  development  of  the  CCDB-East.  IDA’s  task  was  to  identify  the 
characteristics  that  would  make  the  data  most  useful  for  a  technical  analysis.  To  that  end, 
IDA  fonnulated  the  following  research  questions  to  drive  this  study: 

•  What  data  are  necessary  for  the  CCDB-East? 

•  What  processes  are  necessary  for  the  CCDB-East? 

•  What  analytical  capabilities  are  necessary  for  the  CCDB-East? 

•  How  can  (better)  event  data  be  used  to  support  CENTCOM’s  CN  programs? 

B.  Methodology 

IDA  employed  a  multi-faceted  approach  to  this  research  utilizing  qualitative  and 
quantitative  methods.  Analysts  conducted  research  along  three  distinct,  but  mutually 
reinforcing,  lines  of  inquiry: 

1 .  Examine  and  assess  current  data  collection,  validation,  and  analysis  efforts  in 
the  Western  Hemisphere  (WH)  by  observing  the  CCDB  process,  techniques, 
and  use  of  collected  data 


Unlike  the  current  validation  process,  which  reviews  the  content  of  various  reports  to  avoid 
duplication,  IDA  emphasizes  technical  validation  of  event  data  to  ensure  different  sources  of  data  “tell 
the  same  story”  and  that  drug  volumes,  flow,  consumption,  or  other  estimates  of  the  drug  system  are 
reliable  and  accurate. 


2 


2.  Examine  and  assess  existing  data  collection,  validation,  and  analysis  efforts  in 
Southwest  Asia  (SWA)  by  interviewing  United  States  Government  (USG) 
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departments  and  agencies  that  have  a  role  in  drug  data  collection  in  the  region 

3.  Devise  advanced  statistical  techniques  that  address  some  of  the  challenges 
identified  in  the  first  two  phases  and  illustrate  how  they  may  be  used  to  support 
CENTCOM’s  countemarcotics  programs 

The  first  two  lines  of  inquiry  were  presented  in  previously  published  IDA 
documents.4  This  document  addresses  the  third  and  most  analytically  challenging  line  of 
inquiry.  IDA  developed  advanced  statistical  techniques  and  tested  those  techniques  using 
the  UNODC  database  of  Afghan  and  Central  Asian  seizures  and  extracts  from  the  DIA’s 
Significant  Activities  (SIGACT)  reports  contained  in  the  Combined  Information  Data 
Network  Exchange  (CIDNE)  database. 

1.  The  Use  of  United  Nations  (UN)  Data 

Because  UN  data  has  been  systematically  and  methodically  collected  over  time,  it 
represents  the  most  comprehensive  database  of  seizure  events  in  the  region  (though  it  is 
still  only  as  accurate  as  each  member  state’s  contribution).  UN  data  has  the  additional 
benefit  of  having  been  collected  around  Afghanistan,  including  in  Iran  (where  the  largest 
number  of  seizures  are  made).  IDA  compared  data  from  all  other  sources  to  the  UN  data. 
Since  this  comparison  of  other  data  sources  to  UN  data  is  central  to  the  analytic 
methodology  used  in  this  document,  a  discussion  of  that  data  is  provided  here.  The  UN 
data  has  proven  in  the  past  to  have  significant  utility  when  compared  to  highly  detailed 
USG  data  previously  gathered  and  much  of  it  unknown  to  UN  researchers. 

Figure  1  shows  the  physical  distribution  of  the  9,416  seizures  of  opiates  (opium, 
morphine,  and  heroin)  in  and  around  Afghanistan  that  were  analyzed  during  this 
research.3  These  seizures  have  several  properties:  quantity,  time,  and  location.  Usually 
seizures  are  reported  in  one  of  two  ways:  events  in  time  (where  a  seizure  is  an  event)  or 
quantities  in  time  (where  quantity  is  the  mass  or  weight  of  opiates  seized). 


Barry  D.  Crane,  Ashley-Louise  N.  Bybee,  Andrew  Cseko,  Andrea  S.  Pongo,  and  Zachary  S.  Rabold, 
Phase  II  Findings:  Leveraging  Lessons  Learned  to  Inform  Southwest  Asia  Counter-Drug  Efforts,  IDA 
Document  D-4324,  Draft  Final  (Alexandria,  VA:  Institute  for  Defense  Analyses.  May  2011). 

Ibid,  and  Crane  et  al.,  Developing  a  Strategy. 

This  figure  contains  only  preliminary  data  for  2010. 
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Source:  UNODC  and  USG  data 

Figure  1.  Physical  Distribution  of  One  or  More  Events  Where  Opium,  Morphine  or  Heroin 

Were  Seized,  2000-2010 

Figure  2  shows  the  time  distribution  of  the  seizures.  The  rate  of  these  events, 
partitioned  by  different  quantity  bins  (for  example  the  number  of  seizures  in  the  amount 
of  1-10  kilograms  (kgs)  per  unit  of  time)  is  a  stable  indicator  of  risk-taking  activity 
within  the  narcotics  market.  This  insight  is  what  led  IDA  technical  staff — in  collaboration 
with  UNODC  research  analysts  and  a  representative  from  CENTCOM — to  conclude  in 
December  2010  that  the  distribution  of  the  rate  of  drug  seizure  sizes  (e.g.,  kg  seized 
/event/unit  time)can  be  used  to  determine  the  validity  of  data  describing  other  aspects 
(e.g.,  price)  of  the  drug  market  system.  This  fundamental  feature  of  the  drug  trade 
significantly  improves  IDA’s  analyses  of  the  limited  data  available  in  SWA. 
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Source:  UNODC  and  USG  data 

Figure  2.  Temporal  Distribution  of  Events  Analyzed,  2000-2009 

Figures  3  through  6  (extracted  from  the  2011  UNODC  World  Drug  Report) 
illustrate  global  opiate  seizures.  The  regions  inside  the  red  circle  show  the  data  that  IDA 
analyzed  for  this  task.  Figure  3  shows  the  global  seizures  of  heroin  and  morphine  and 
Figure  4  shows  the  totals  for  those  seizures  in  metric  tons  through  2009. 


Figure  3.  Physical  Distribution  of  Heroin  and  Morphine  Data  Analyzed  from  Map  10, 
“Seizures  of  Heroin  and  Morphine,  2009  (Countries  and  Territories  Reporting  Seizures  of 

More  than  10  kg)” 
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Source:  UNODC,  World  Drug  Report ,  2011,  63 

Figure  4.  Temporal  Distribution  of  Heroin  and  Morphine  Data  Analyzed  from  Figure  29, 
“Global  Seizures  of  Heroin  and  Morphine:  1999-2009” 


Figure  5  shows  the  physical  distribution  of  global  seizures  of  opium  and  Figure  6 
shows  the  totals  for  those  seizures  in  metric  tons  through  2009. 
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Figure  5.  Physical  Distribution  of  Opium  Data  Analyzed  from  Map  11,  “Opium  Seizures  in 

Asia,  2009” 
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Figure  6.  Temporal  Distribution  of  Opium  Data  Analyzed  from  Figure  30,  “Global  Seizures 

of  Opium:  1999-2009”6 


C.  Document  Outline 

Portions  of  this  study  are  predicated  on  decades  of  previous  IDA  research  focused 
on  the  cocaine  market  in  the  WH.7  This  research  provided  many  of  the  foundational 
principles  upon  which  this  study  was  built.  This  document  synthesizes  that  research  with 
the  findings  from  this  study  to  fonnulate  recommendations  for  SWA.  Chapter  2  of  this 
document  is  organized  in  line  with  the  research  questions  presented  on  page  3.  Section  A 
of  Chapter  2  presents  findings  about  data  that  should  be  contained  in  a  CCDB-East  and 
some  of  the  challenges  associated  with  that  data.  Section  B  of  Chapter  2  describes  the 
processes  and  statistical  techniques  that  are  necessary  for  the  proper  interpretation  of  that 
data.  Section  C  of  Chapter  2  presents  the  analytical  capabilities  that  a  CCDB-East  should 
have  in  order  to  best  assist  users.  Section  D  describes  how  a  database  can  be  used  to 
support  CENTCOM’s  CN  programs.  Chapter  3  offers  recommendations  and  Chapter  4 
presents  conclusions. 


IDA  analyses  used  more  than  95%  of  about  400-600  metric  tons  of  opium  seized  per  year. 
See  Appendix  B,  “Analysis  of  the  Cocaine  Market.” 
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2.  Research  Findings 


A.  What  Data  Are  Necessary? 

IDA’s  task  was  to  identify  the  characteristics  that  would  make  the  data  most  useful 
for  a  technical  analysis.  For  this  aspect  of  the  task,  IDA  did  not  examine  the  specific 
fields  that  would  be  necessary  in  the  CCDB-East.  This  is  a  function  already  being 
performed  by  DIA. 

1.  Raw,  not  averaged,  data  are  necessary  for  more  accurate  analyses. 

Previous  IDA  research  has  shown  that  power  law  distributions  (where  many  events 
occur  at  small  values  and  few  events  occur  at  large  values,  and  there  is  a  consistent 
pattern  that  systematically  relates  incidence  rates  to  changes  in  value  sizes)  characterize 
many  components  of  the  illicit  drug  trade.  The  adherence  of  drug  data,  including 

o 

quantities  seized,  consumption,  and  the  price  of  a  given  volume,  to  these  known 
statistical  distributions  has  one  critical  implication  for  interpreting  the  data:  the  use  of  the 
mean  (average)  as  an  estimate  of  any  power  law-like  distribution  may  be  extremely 
misleading — both  for  summarizing  a  single  data  set  and  for  characterizing  trends 
exhibited  by  multiple  data  sets  over  time.  Medians,  on  the  other  hand,  offer  a  more 
accurate  and  stable  representation  of  the  entire  system  of  analytic  products.9 
Unfortunately,  most  reporting  to  policy  makers  typically  comes  in  the  form  of  monthly 
averages  (i.e.,  means),  which  can  misrepresent  the  facts. 

Depending  on  the  sample  size,  the  divergence  between  the  mean  and  median  can  be 
quite  large. 10  Figure  7  shows  a  typical  comparison  between  means  and  medians  using  UN 
seizure  data,  which  produces  a  highly  asymmetric  and  skewed  distribution.  The  M 


See  Appendix  A,  “Models  and  Simulations,’’  and  Appendix  B,  “Analysis  of  the  Cocaine  Market,”  for 
more  detail. 

Barry  D.  Crane,  A.  Rex  Rivolo,  and  Gary  C.  Comfort,  An  Empirical  Examination  of  Counterdrug 
Interdiction  Program  Effectiveness,  IDA  Paper  P-3219  (Alexandria,  VA:  Institute  for  Defense 
Analyses,  1997).  The  first  arguments  are  presented  that  for  unknown  (especially  power  law-like) 
distributions  only  the  medians  might  converge.  Previous  engineering  research  of  a  similar  technical 
problem  when  analyzing  the  signal-to-noise  and  detection  performance  of  over-the-horizon  radars  were 
well  characterized  by  median  statistical  representations.  Testing  these  analyses  over  ten  years  confirm 
the  practical  use  of  these  techniques. 

For  a  more  detailed  description  of  the  inflationary  effects  of  a  statistical  mean  and  the  use  of  the 
median  value,  see  Appendix  A,  “Models  and  Simulations.” 
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(medians)  and  A  (averages/mean)  are  shown  for  actual  heroin  and  opium  data  (in  red  and 
blue  respectively),  with  each  median  being  substantially  less  than  its  average/mean 
counterpart  for  lower  quantities  seized. 


Quantity  Seized  (kg) 


Source:  UNODC 

Note:  M=median.  A=average  (mean).  Heroin=red.  Opium=blue 

Figure  7.  A  Small  Sample  of  UNODC  Heroin  and  Opium  Seizures  Showing  Overall 
Averages  and  Medians  for  All  UNODC  Data 

Figure  8  illustrates  the  significantly  different  distributions  between  the  UN’s 
opium11  event  data  (2,606  total  events  collected  between  2000  and  2010)  and  the  USG’s 
chance  data  (295  total  events  collected  between  2008  and  2010).  Chance  data  refers  to 
drug  seizures  made  during  a  military  operation  that  was  not  planned  or  intended  to  be  a 
CN  operation.  Due  to  limited  collection  assets,  capabilities,  or  systematic  data  collection 
procedures,  most  USG  event  data  falls  into  this  category.  Logarithmic-scaled  axes  (log  of 
the  events  and  the  log  of  the  quantities)  are  used  to  show  the  extraordinary  range  of  all 
data  quantities  and  the  stability  of  the  seizure  distributions. 


Opium  data  was  used  over  heroin  data  due  to  its  greater  sensitivity  to  enforcement  actions.  See 
Thomas  Pietschman,  “Price-setting  Behaviour  in  the  Heroin  Market,”  UNODC  Bulletin  on  Narcotics: 
Illicit  Drug  Markets  LVI,  nos.  1  and  2  (2004):  105. 
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Note:  mt=metric  tons  and  kg=kilograms.  The  box  displays  the  region  where  USG  event  data  is 
significantly  under-sampled.  The  flag  boxes  show  typical  number  of  events  at  each  quantity. 

Figure  8.  A  Comparison  of  UN  Opium  and  USG  Event  Data  and  Underlying  Distribution 


A  sampling  bias  causes  the  difference  in  these  distributions.  UN  data  is  collected  in 
a  systematic  fashion,  resulting  in  a  proportionate  number  of  events  collected  at  both  the 
low  and  high  ends  of  the  distribution  curve.  On  the  other  hand,  USG  chance  data 
significantly  under  samples  the  low  end  of  the  distribution  curve  (small  events)  because 
arbitrary  reporting  thresholds  are  too  high  to  capture  the  majority  of  small  seizure  events. 
This  is  why  the  two  curves  are  consistent  with  each  other  for  seizures  greater  than  100  kg 
while  there  is  significant  deviation  for  events  less  than  100  kg  (see  the  box  in  Figure  8). 
In  other  words,  the  UN  consistently  has  two  times  as  many  events  recorded  for  events 
larger  than  100  kg  while  it  has  approximately  230  times  as  many  events  recorded  for  1  kg 
events  (see  call-out  boxes  in  Figure  8). 

The  medians  and  means  depicted  in  Table  1  can  be  calculated  based  on  these  data. 

Both  the  USG  median  and  mean  overestimate  trafficking  and  are  about  3.5  times  larger 

12 

than  the  associated  estimate  derived  from  the  more  complete  UN  data  set. 


Incidentally,  the  average  cocaine  seizure  value  taken  from  CCDB  data  is  about  20  times  the  median 
cocaine  seizure  value. 
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Table  1.  Gross  Descriptors  of  UN  and  USG  Data 


Total  (mt) 

Average (kg) 

Median  (kg) 

UN  Data 

278 

99 

12 

USG  Data 

105 

350 

40 

The  events/kilogram  format  also  allows  analysts  to  examine  the  rate  of  trafficking 
of  various  quantities.  Figure  8  illustrates  that  high-risk  events  (involving  larger 
quantities)  occur  at  a  slower  rate  while  low-risk  events  (involving  smaller  quantities) 
occur  at  a  much  faster  rate.  This  is  consistent  with  the  principle  of  equal  risks  for  equal 
returns  which  is  described  in  more  detail  under  research  finding  4.  However,  this 
generally  accepted  principle  is  not  observed  in  the  highlighted  portion  of  the  USG  data 
(for  small  seizures  of  less  than  100  kg),  providing  further  evidence  that  USG  data  under 
samples  small,  low-risk  events. 

The  implications  of  this  finding  are  multi-fold.  As  it  is  currently  collected,  USG 
data  cannot  be  used  to  calculate  accurate  flow  estimates,  including  averages  and  medians, 
due  to  the  bias  that  results  from  the  USG’s  unsystematic  collection  procedures.  This 
sampling  bias  also  affects  USG  estimates  for  other  important  characteristics  of  the  drug 
trade,  such  as  quantities  trafficked  or  surrogate  prices.  Monthly  averages  derived  from 
current  USG  data,  such  as  those  typically  reported  to  policy  makers,  likely  contain  errors 
and  may  be  misleading  (e.g.,  overestimating  enemy  capabilities  and  making  it  difficult  to 
accurately  assess  trends).  Furthennore,  the  sample  average  will  vary  dramatically  over 
time.  Tracking  sample  averages  over  regularly  spaced  time  periods  can  obscure  real 
trends  and  even — in  the  short  term — create  a  false  appearance  of  a  trend.  Even  with  the 
sampling  bias,  more  stable  and  operationally  useful  analytic  descriptions  of  drug  data  can 
be  achieved  using  the  median  value  for  the  sample. 

Due  to  the  operational  environment  in  SWA,  where  there  are  limited  personnel, 
platfonns,  and  assets  available  for  drug  event  data  collection,  it  is  unrealistic  to  expect  to 
collect  a  large  sample  size  of  opiate  data.  In  lieu  of  a  very  large  sample  (data  set)  for 
SWA  opiates,  analysts  must  ensure  that  they  overcome  some  of  the  challenges  associated 
with  small  sample  sizes.  To  compound  the  challenge  of  limited  data,  small  numbers  of 
seizure  events  (which  are  very  common)  over  short  time  intervals  can  be  under¬ 
represented  in  a  sample  (for  example,  because  of  an  arbitrary  reporting  threshold  that  is 
too  high  to  capture  the  majority  of  small  seizure  events).  Under  these  circumstances, 
using  the  larger  derived  mean  value  from  these  small  samples  to  estimate  a  typical  value 
for  a  drug  seizure  is  likely  to  result  in  a  grossly  inflated  estimate. 

2.  It  is  essential  to  maintain  the  integrity  of  separate  data  sets. 

In  order  to  compare  and  validate  data  sets  accurately,  the  data  sets  themselves  must 
reflect  systematic  data  collection  procedures  that  are  applied  consistently  over  time.  IDA 
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technical  experts  found  that  when  data  sources  are  combined  into  a  single  data  set,  there 
are  invariably  differences  in  the  raw  data  that  distort  the  time-dependent  infonnation — 
particularly  when  the  criteria  for  data  inclusion  changes,  making  relative  measurements 
difficult.  For  example,  combining  UN  and  USG  seizure  event  databases  eliminates  the 
potential  for  analysts  to  calculate  estimates  and/or  extract  valuable  insights  from  the 
UN’s  systematically  collected  data. 

The  following  example  shows  opium  cultivation  and  production  estimates  from 
USG  (specifically  the  Central  Intelligence  Agency,  (CIA))  and  UN  (UNODC,  World 
Drug  Report,  2011 )  data.  By  comparing  these  two  independent  data  sets,  it  is  possible  to 
estimate  what  may  be  termed  the  “measurement  error.”  This  measurement  error  is 
actually  the  difference  between  the  two  data  sets  that  may  occur  as  a  result  of  different 
collection  methodologies  or  biases  inherent  to  the  data  collection  process.  Tracking  the 
measurement  error  is  a  useful  way  to  detennine  the  underlying  accuracy  of  the  data  sets, 
i.e.,  if  differences  in  measurement  are  consistent  over  time,  then  one  can  place  higher 
confidence  in  the  accuracy  of  the  data.  It  is,  of  course,  possible  that  both  data  sets  may 
have  inherent  errors  in  measurement.  However,  these  differences  in  measurement  are 
inconsequential  if  both  data  sets  display  the  same  trends.  Determining  that  two  data  sets 
are  consistent  with  each  other,  i.e.,  they  “tell  the  same  story,”  is  a  valuable  validation 
technique.  Ultimately  the  most  valuable  insights  are  derived  from  analyzing  the  overall 
trends  of  the  data. 

Figure  9  compares  USG  and  UN  cultivation  estimates,  including  a  “best  fit  to  the 
data”  curve  for  each  data  series.  Using  this  presentation,  one  can  visually  observe  the 
measurement  error,  or  difference  in  measurement,  which  is  the  difference  between  the 
dotted  lines.  In  this  example  there  is  a  measurement  difference  of  approximately  +/- 
25,000  hectares  before  2005  and  approximately  +/-  5,000  hectares  since  2005. 13  The 
converging  data  points  since  2005  reflect  recent  improvements  in  estimates. 


After  2004,  significant  efforts  were  made  to  discuss  technical  means  for  generating  cultivation 
estimates.  These  most  likely  reduced  the  measurement  error. 
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Source:  USG  crop  estimate,  and  UNODC,  World  Drug  Report,  2011 

Figure  9.  Estimates  of  Afghan  Opium  Cultivation  (hectares) 


Similarly,  Figure  10  compares  production  estimates  since  1996.  These  estimates 
may  vary  slightly  due  to  differences  in  cultivation  and  yield.  An  approximate 
measurement  error  (or  difference)  of  +/-  1000  metric  tons  (mt)  exists  prior  to  2004. 
Estimates  appear  to  be  very  close  during  the  2004-2006  time  frame,  but  they  diverged 
again  after  2006.  The  main  reason  for  these  variations  in  measurements  is  ongoing 
disagreement  over  the  calculation  of  production  levels  based  on  cultivation  and  the  yields 
of  opium  gum  per  hectare. 
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Source:  USG  crop  estimate,  and  UNODC,  World  Drug  Report,  2011 

Figure  10.  Estimates  of  Afghan  Opium  Production  (Metric  Ton) 


While  this  example  highlights  the  importance  of  maintaining  the  integrity  of 
separate  data  sets  so  that  each  may  be  checked  for  consistency  (i.e.,  validated),  the 
following  example  using  cocaine  data  illustrates  the  danger  of  combining  different  time 
series  data.  Figure  1 1  depicts  two  sets  of  cocaine  production  estimates  over  a  common 
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time  span.  The  green  data  points  plot  the  estimates  of  the  Colombian  National  Police 
(CNP)  (generated  by  the  Integrated  Crops  Monitoring  System,  known  by  its  Colombian 
acronym  SIMCI).  The  red  data  points  plot  the  USG  estimates.  The  two  data  sets  used 
different  data  collection  and  analysis  approaches,  but  individually  their  methodologies 
were  applied  consistently  over  time  and  yielded  the  same  general  trends,  thus  validating 
each  other.  Therefore,  while  their  respective  absolute  estimates  vary — with  the  USG 
always  reporting  lower  annual  values — each  set  of  results  portrays  the  same  general 
declining  trend  in  cocaine  production.  The  dotted  black  line  shows  the  UNODC  estimates 
of  Colombian  cocaine  production  which  combines  USG  and  SIMCI  data. 
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Source:  Data  provided  by  the  Colombian  National  Police  (CNP)  and  the  USG 

Figure  11.  Colombian  Cocaine  Production:  Separate  Time  Series 


The  UN  used  the  USG  data  until  2004  and  then  switched  to  SIMCI  data  when  that 
program  matured.  As  a  result,  the  UNODC  representation  of  the  data  shows  a  relatively 
steady  level  of  cocaine  production  prior  to  2008. 

The  UN  combined  data  set  display  implied  that  Colombian  production  control  was 
ineffective,  since  according  to  the  UNODC  representation  of  the  data,  production  was  not 
affected  by  the  counternarcotics  campaign.  Prior  to  2003  the  UN  varied  the  ratio  of 
attributing  values  from  USG  and  UN  estimates  in  production  and  cultivation  in  order  to 
yield  a  single  value  for  production.  In  some  cases  the  UN  used  USG  production  values 
applied  to  UN  cultivation  figures.  Combining  these  values  made  it  appear  that  there  was 
no  change  in  production,  though  in  reality  production  control  was  effective  as  evidenced 
by  the  two  distinct  and  separate  data  sets  provided  by  the  U.S.  and  Colombian 
governments.  This  example  illustrates  the  importance  of  examining  independent  data  sets 
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separately  to  validate  each  other  and  corroborate  conclusions.  It  also  illustrates  the  danger 
of  consolidating  disparate  data  sets  prior  to  analysis. 

Whereas  the  Afghan  opium  example  (Figures  9  and  10)  shows  both  absolute  values 
and  trends  that  are  similar,  the  Columbian  cocaine  data  (Figure  11)  shows  trends  that  are 
similar,  but  absolute  numbers  that  are  not.  This  suggests  a  much  larger  measurement 
error  and  as  such,  the  two  Columbian  cocaine  data  sets  should  not  be  combined,  as  the 
ability  to  measure  quality  and  absolute  numbers  is  lost. 

CCDB  data  collection  procedures  for  cocaine  have  continuously  evolved  since  the 
inception  of  the  Interagency  Assessment  of  Cocaine  Movement  (IACM)  in  1991.  The 
CCDB,  however,  never  formally  incorporated  many  of  the  other  independent  data 
sources  (UN,  Colombian,  Mexican,  and  U.S.  Drug  Enforcement  Administration  (DEA) 
data)  needed  to  validate  and  assist  in  interpreting  the  CCDB  data  sets.  More  importantly, 
there  has  been  no  concerted  effort  to  ensure  systematic  data  collection,  nor  to  control 
changes  over  time  in  the  assignment  of  confidence  levels  to  the  CCDB  data.  For  example, 
while  the  IACM  includes  confirmed  drug  seizures,  it  also  includes  events  relating  to 
unconfirmed  reports  of  drug  movements.  Unconfirmed  reports  may  be  vague  and  even 
speculative,  generating  large  period-to-period  reporting  variances  in  IACM  data.  The 
rules  for  assigning  confidence  levels  to  those  data  can  be  changed  year-to-year  making  it 
hard  to  compare  confidence  levels  over  time.  Assignment  of  high  confidence  to  reports  of 
one  type  (e.g.,  estimated  shipping  volumes)  in  one  year  and  low  confidence  to  the  same 
type  of  data  in  the  next  year  inhibits  use  of  the  data  in  temporal  analyses. 

IDA  finds  that  an  effective,  event-driven  database  must  insure  the  integrity  of  its 
separate  data  inputs  by  accounting  for  differences  in  collection  methodology, 
measurements,  etc.,  for  each  data  set.  The  data  collection  methodology  must  be  consistent 
within  any  given  data  set.  Changing  data  sources,  contributors,  or  other  important 
collection  features  will  affect  the  way  events  are  characterized,  measured,  or  otherwise 
recorded.  Comparison  across  independent  data  sets  allows  the  analyst  to  detect  these 
changes.  The  difficulty  for  the  CCDB-East  is  the  lack  of  independent  data  sets  with 
which  to  perform  this  validation. 

3.  Systematized  data  collection  methods  yield  more  reliable  information. 

Based  on  lessons  learned  from  decades  of  operations  in  the  WH,  effective  CN  data 
collection  and  information  sharing  requires  interagency  participation  and  active,  effective 
coordination.  Coordination  insures  that  needed  and  relevant  infonnation  is  being  shared 
with  analysts  who  seek  it.  There  are  several  entities  currently  collecting  drug  data  in 
Afghanistan,  including  the  U.S.  military,  the  DEA,  and  international  partners  in  the 
International  Security  Assistance  Force  (ISAF).  Although  only  the  DEA  is  formally 
tasked  with  a  CN  mission,  about  90%  of  drug  event  reporting  used  by  DIA  comes  from 
military  sources.  IDA  researchers  also  learned  that  ISAF  personnel  are  making  numerous 
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chance  drug  seizures  during  operations,  but  much  of  this  data  may  not  be  provided  to  the 
U.S.  CN  community  and  its  allies  for  security  reasons.  Similarly,  U.S.  Customs  and 
Border  Protection  and  U.S.  Immigration  and  Customs  Enforcement  personnel  have  been 
collecting  data  on  precursor  chemicals — infonnation  that  is  eagerly  sought  by  the  rest  of 
the  CN  community,  but  that  remains  unavailable  to  them  because  of  a  lack  of 
coordination. 14 

Coordination  of  data  collection  can  also  facilitate  the  use  of  technical  guidance  to 
inform  collection  procedures.  For  example,  this  document  emphasizes  the  importance  of 
using  lower  reporting  thresholds  (1  kg  versus  100  kg)  that  more  accurately  reflect  the  rate 
of  market  activity,  since  smaller  quantities  of  drugs  are  most  frequently  trafficked  in 
SWA.  Consistently  applied  collection  and  reporting  guidance  also  helps  to  systematize 
drug  data  collection  by  requiring  that  specific  fields  of  information  be  included  (for 
example,  seizure  geo-coordinates  or  event  confidence  levels). 

It  is  difficult  but  possible  to  coordinate  contributions  of  drug  data  from  law 
enforcement  (LE),  military,  and  allied  personnel  in  Afghanistan.  Voluntary  cooperation 
will  be  easier  if  a  system  is  established  to  automatically  ‘push’  data  from  interagency 
partners  to  a  centralized  database,  rather  than  having  database  administrators  ‘pull’ 
infonnation  from  each  partner,  especially  since  partners  will  not  always  be  aware  of  the 
specific  data  potentially  available  from  other  partners. 

Current  U.S.  drug  data  collection  efforts  in  SWA  are  still  nascent  and  coordination 
of  data  collection  is  very  important  due  to  limited  assets,  platforms,  personnel,  and 
capabilities  available  in  the  region.  If  military  resources  are  scaled  back,  reporting  will  be 
even  more  limited.  However,  even  limited  data  may  still  be  useful  for  analyses  if  it  can  be 
validated  and  a  determination  made  regarding  its  quality. 

B.  What  Processes  Are  Necessary? 

In  order  to  interpret  data  correctly,  it  is  essential  that  the  data  be  of  the  highest 
confidence  and  that  they  have  been  validated  to  ensure  their  accuracy.  The  following 
processes  must  be  applied  to  ensure  the  quality  of  the  data  contained  in  the  CCDB-East. 

1.  Event  data  should  be  categorized  by  confidence  levels  before  analysis. 

In  observing  the  CCDB  process,  IDA  found  that  one  of  its  greatest  strengths  is  its 
deliberative  process  to  vet  the  assigned  confidence  levels  of  data  in  the  database.  Events 
are  categorized  into  one  of  four  confidence  levels  (previous  iterations  of  the  confidence¬ 
assigning  process  had  three  categories  of  confidence  levels).  Physical  seizures  are  rated  at 


For  more  detail,  see  Crane  et  al.,  Phase  II  Findings:  Leveraging  Lessons  Learned  to  Inform  Southwest 
Asia  Counter-Drug  Efforts. 
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the  highest  level  of  confidence  and  are  categorized  as  “Confirmed”  or  “C.”  Events  that 
cannot  be  confirmed  by  physical  evidence,  but  have  been  either  visibly  observed  or 
described  by  multiple,  independent,  and  corroborating  sources  are  categorized  as 
“Substantiated”  or  “S.”  Lastly  suspected  events  assumed  to  involve  illicit  drugs — such  as 
open-source  press  reports  or  single  intelligence  reports — in  the  CCDB  are  rated  at  the 
“Suspect”  or,  more  recently,  “IP”  or  “2P”  levels  of  confidence  (with  IP  indicating  a 
higher  level  of  confidence  than  2P).  Earlier  representations  of  the  confidence  levels  had 
only  one  category  for  the  “P”  data  contained  in  what  is  now  IP  and  2P  data.15  Figure  12 
illustrates  this  confidence-ranking  process. 


Early  analysis  of  P  data  suggested  that  it  was  only  accurate  about  5%  to  10%  of  the  time  based  upon 
quantitative  comparisons.  This  type  of  analysis  does  not  occur  today. 


18 


Lift  Updated  11/26JDB 


Source:  CCDB  User’s  Guide,  Work  in  Progress,  June  2010,  1916 

Figure  12.  CCDB  Confidence  Ranking  Flow  Chart 


IDA  finds  that  this  practice  of  designating  confidence  levels  is  critical,  since  the  use 
of  P-level  data,  in  some  analyses,  can  drastically  skew  results  and  lead  to  erroneous 
conclusions.  For  example,  Figure  13  shows  reported  cocaine  movements  based  on 
Confirmed  and  Substantiated  data  (in  red  and  green  respectively)  compared  with  the 
reported  cocaine  movement  based  on  low-confidence  P-level  data  (in  purple). 
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The  IP  and  2P  data  illustrated  here  are  referred  to  collectively  as  “P  data”  in  subsequent  sections. 
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Source:  IACM,  2010 

Figure  13.  Reported  Cocaine  Movements  by  Confidence  Level 


A  significant  discrepancy  emerges  from  this  chart,  with  the  P  data,  and  to  a  certain 
extent  the  S  data,  differing  from  the  C  data  (particularly  in  the  2006-2007  and  2008- 
2009  time  frames).  Given  this  discrepancy,  IDA  recommends  testing  the  P  data  as  part  of 
the  validation  process.  This  chapter  presents  several  proven  validation  techniques  to 
determine  whether  P  data  are  consistent  with  other,  higher-confidence  data.  If  the  P  data 
is  found  to  be  inconsistent  with  other  data  sources  (such  as  price  or  purity)  then  the 
unexplained  behavior  of  the  data  should  cast  doubt  on  its  utility  for  policy  analysis. 
Ultimately  its  inclusion  in  official  estimates,  as  it  is  currently  presented  in  the  quarterly 
IACM,  may  have  the  effect  of  inflating  the  total  estimated  quantity  of  cocaine 
movements  and  overestimating  enemy  capabilities. 

Nevertheless,  capturing  P-level  data  in  the  CCDB  is  still  a  valuable  practice  that 
should  be  continued.  Moreover,  particular  attention  should  be  paid  to  P  data  that  may  be 
“promoted”  to  higher  confidence  levels  when  new  data  becomes  available.  Including  all 
data,  ranging  from  the  most  reliable  to  the  least,  allows  analysts  to  use  the  data  many 
ways.  For  example,  some  agencies  rely  on  the  highest  quality  data  to  drive  resourcing 
decisions — including  the  allocation  of  their  assets — but  still  value  low-confidence  data 
for  context  and  for  understanding  the  potential  scope  of  trafficking  activity.  The  ability  to 
extract  data  of  different  confidence  types  for  different  analyses  is  a  great  strength  of  the 
CCDB. 

2.  Event  data  may  be  validated  using  time  series  plots. 

Time  series  plots  are  important  validation  tools  to  understand  the  drug  market  as  a 
system  because  they  allow  analysts  to  compare  different  variables  over  time.  When 
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linked  together,  time  series  comparisons  show  analysts  how  complex  systems  behave  and 

1 7 

respond  to  the  CN  campaign  or  other  changes  in  the  system’s  environment. 

For  example,  Figure  14  illustrates  opium  prices  reported  by  traders  in  Kandahar  and 
Nangarhar,  Afghanistan  between  March  1997  and  June  2011.  In  this  case,  the  time  series 
data  show  that  opium  prices  are  highly  responsive  to  external  shocks  to  the  drug 
trafficking  system  and  also  to  expectations  of  future  supply.  The  dramatic  price  spike 
observed  in  mid-2001  was  caused  by  the  Taliban  ban  on  poppy  cultivation.  Immediately 
following  the  September  1 1  attacks  on  the  United  States,  Afghan  traders  dumped  stocks 
of  opium  in  expectation  of  retaliatory  attacks.  The  drop-off  shown  during  2004  was 
caused  by  high  levels  of  opium  production  and  the  lax  implementation  of  a  poppy 
cultivation  ban  previously  announced  by  the  Karzai  government.  As  a  result,  supply 
increased  and  prices  quickly  dropped. 
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Pietschman,  “Price-setting  Behaviour,”  119. 
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The  dramatic  price  spike 
observed  in  mid- 2001  was 
caused  by  the  Taliban  ban  on 
poppy  cultivation. 
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High  levels  of  opium  production  and 
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Figure  14.  Opium  Prices  Reported  by  Traders  in  Kandahar  (Green)  and  Nangarhar  (Orange) 

Vertical 


The  steady  but  moderate  price  decline  between  2006  and  2009  reflects  very  high 
levels  of  global  and  Afghan  opium  production  at  that  time.  The  chart  shows  a  turnaround 
in  average  prices  in  2010  caused  by  a  serious  drop-off  in  opium  supply  resulting  from  a 
blight  that  eradicated  approximately  61%  to  87%  of  opium  yields  in  disease-affected 
fields.  Even  though  the  same  approximate  number  of  hectares  of  poppy  was  cultivated  in 
2010  as  in  2009,  the  blight — in  addition  to  the  effects  of  frost  and  drought — contributed 
to  the  ongoing  increase  in  Afghan  opium  prices. 19 

3.  Event  data  may  be  validated  using  independent  variables  that  are  known  to 
indirectly  corroborate  event  data. 

Validating  drug  event  data  by  testing  them  against  independent  variables,  such  as 
price  or  purity,  is  a  second  useful  technique.  Since  purity  levels  are  known  to  correlate 
with  actual  flow  estimates,  purity  levels  over  time  should  move  consistently  with 
estimates  of  flow.  If  not,  this  could  reflect  poor  quality  data,  or  have  a  more  significant 


UNODC,  Afghanistan  Opium  Sun’ey  2010,  Summary’  Findings,  September  2010,  2,  16,  22. 


22 


implication,  such  as  the  diversion  of  drugs  or  other  developments  that  should  be 
investigated  further. 

Figure  15  shows  a  comparison  between  the  volume  of  U.S.  cocaine  movement 
(using  only  ‘Confirmed’  and  ‘Substantiated’  data  from  the  CCDB)  and  the  general  purity 
of  cocaine  (measured  by  the  DEA).  The  blue  line  shows  the  higher  confidence 
movements  of  cocaine  while  the  red  line  shows  the  mean  purity  of  the  1  kg  seizures 
contained  within  a  particular  time  interval  (i.e.,  the  quality  of  product  being  seized,  either 
cocaine  base  or  powder). 
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Source:  CCDB  and  STRIDE  databases 
Note:  TZ=Transit  Zone 

Figure  15.  Cocaine  Movement  (as  Reported  by  the  CCDB)  and  Purity  (as  Measured  by  the 

DEA  in  the  STRIDE  Database)  2001-2010 


This  graph  shows  how,  in  general,  purity  levels  track  with  the  volume  of  cocaine 
being  moved,  assuming  constant  demand.  Interestingly,  because  this  graph  uses  C  and 
S  data  for  movement — and  demonstrates  their  validity  by  comparing  them  with  purity 
data — one  can  infer  inconsistencies  in  P-level  data  (see  Figure  13,  Reported  Cocaine 
Movements  by  Confidence  Level).  Because  IP  and  2P  data  (previously  referred  to  as 
simply  P  data)  are  not  consistent  with  C  data  (particularly  between  2006  and  2008  where 
P  data  indicates  an  increasing  trend  while  the  C  data  indicates  a  declining  trend)  one 
could  hypothesize  that  that  P-level  data  does  not  generate  an  accurate  depiction  of 
cocaine  movement.  Moreover,  as  Figure  15  shows,  purity  levels  in  the  2008  time  frame 


Spikes  in  the  blue  line  correspond  with  various  CN  operations  when  additional  seizures  were  made. 
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do  not  corroborate  the  P  data,  providing  further  evidence  that  P  data  are  not  accurate 
representations  of  real  cocaine  movements. 

Purity  is  a  valuable  independent  variable  to  use,  not  only  because  the  DEA  collects 
hard  and  reliable  data  on  it  (as  opposed  to  estimates),  but  also  because  it  facilitates  a 
better  description  of  seizures  beyond  quantities  of  unknown  quality.  Through  its  System 
to  Retrieve  Information  from  Drug  Evidence  (STRIDE)  database,  the  DEA  captures 
purity  levels,  prices,  and  other  market  data  collected  in  the  continental  U.S.  (CONUS) 
when  undercover  agents  pose  as  consumers  and/or  resellers  of  illicit  substances  and 
purchase  drugs.  Thus,  STRIDE  is  a  particularly  useful  data  source  to  validate  CCDB  data 
because  it  provides  an  additional  independent  variable. 

4.  Event  data  may  be  validated  by  comparing  it  with  the  underlying  functional 

distributions  (power  laws). 

IDA’s  research  has  identified  one  critical  feature  shared  by  most  components  of  a 

drug  system — their  behavior  is  consistent  with  established  statistical  patterns  known  as 

power  laws.  A  power  law-like  function  is  a  mathematical  relationship  between  two 

values  where  the  frequency  of  an  event  (e.g.,  a  drug  seizure)  varies  as  a  power  of  an 

attribute  of  the  event  (e.g.,  the  size  of  the  drug  seizure).  That  is  to  say,  the  frequency  of 

drug  trafficking  events  for  smaller  quantities  of  a  drug  is  higher  than  the  frequency  of 

events  for  large  quantities.  While  this  is  inherently  intuitive,  the  “power”  of  power  laws 

are  that  they  capture,  in  a  systematic  way,  the  rate  at  which  these  incidences  decrease 

over  a  great  range.  The  slope  that  is  generated  when  original  scales  are  changed  to  log- 

2 1 

log  is  a  valuable  finding  for  each  data  set.' 

IDA  analysts  have  established  that  these  unambiguous  distributions,  or  power  laws, 
provide  useful  insights  into  illicit  drug  markets.  They  are  ubiquitous  across  most  types  of 
drug  data:  transaction  sizes,  user  rates,  price  ratios,  time  intervals  between  seizure  events, 
and  the  likelihood  that  criminals  will  traffic  drugs  given  a  certain  perceived  likelihood  of 
arrest.  '  Further,  the  more  rigorous  the  data  collection  procedures,  the  more  likely  the 
data  set  will  follow  the  expected  pattern.  As  such,  power  laws  provide  a  valuable  baseline 
against  which  to  compare  actual  event  data.  Because  the  distribution  of  drug  data  is  so 
consistent  with  power  laws,  event  data  inconsistent  with  a  power  law  formulation  could 
indicate  potentially  suspect  or  revealing  information,  such  as  a  new  trafficking  behavior, 
missing  infonnation,  or  a  flawed  collection  procedure.  These  types  of  anomalies  should 
prompt  analysts  to  conduct  further  research  in  order  to  account  for  these  deviations. 


The  UNODC  Director  requested  that  IDA  brief  UNODC  analysts  on  these  emerging  techniques  in 
order  to  potentially  improve  the  World  Drug  Report. 

For  more  detail  see  Appendix  B,  “Analysis  of  the  Cocaine  Market.” 
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The  use  of  power  law  distributions  to  validate  opiate  data  was  discussed  during 
technical  meetings  among  IDA  staff,  UN  researchers,  and  the  government  sponsor.  IDA 
presented  analyses  that  showed  that  the  UN’s  systematically  collected  data  follow  the 
functional  distribution,  whereas  non-systematically  collected  data  (e.g.,  chance  data 
currently  collected  by  the  USG)  do  not.  This  process  of  comparing  a  limited  sample  of 
data  with  a  power  law  serves  as  an  independent  check  of  the  quality  of  the  data  and, 
therefore,  a  valuable  validation  method. 

Figures  16  and  17,  based  on  UN  data,  illustrate  the  applicability  and  utility  of  power 
law  representations.  This  is  a  small  subset  of  the  UN  data,  excerpted  to  show  the  means 
and  medians. 


Heroin;  RJ  *  0.77 
Avg  »  17.0;  Media  =  1.2 


Quantity  Seized  (kg) 

Source:  UNODC 

Figure  16.  Number  of  Binned  Seizure  Events  per  kg  for  Heroin  and  Opium  and 
Corresponding  Averages  and  Medians 

Presenting  the  data  in  a  graph  makes  it  easier  to  visualize  the  statistical  distribution 
of  these  seizures.  In  Figure  16,  the  seizure  size  data  are  partitioned  into  quantity  bins 
(e.g.,  Bin  =  1  to  50  kg),  with  the  middle  of  each  bin  represented  by  a  point  in  the  graph. 
The  total  number  of  seizures  recorded  for  each  bin  is  divided  by  the  nominal  size  (in 
kilograms)  associated  with  that  bin,  and  the  resultant  pairs  of  values  (i.e.,  size  and 
normalized  counts)  constitute  the  points  displayed  in  the  figure.  Both  the  averages/means 
(A)  and  the  medians  (M)  are  shown.  The  order  of  magnitude  between  averages  and 
medians  shows  a  highly  asymmetric  distribution.  The  opium  average  of  about  100  kg 


There  also  appears  to  be  significant  under  sampling  of  the  smaller  values,  which  results  in  a  distorted 
distribution. 
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shows  that  a  few  very  large  seizures  drive  the  average  with  most  events  involving  smaller 
seizure  quantities.  Because  the  data  are  displayed  on  ordinal  scales,  the  graph  omits  the 
many  more  extreme  data  points.  The  opium  data  scale  maximum  range  is  to  100  kg, 
where  the  underlying  data  set  contains  at  least  one  reported  seizure  of  about  10,000  kg. 

When  the  sample  averages  and  medians  are  as  divergent  as  they  are  in  Figure  17, 
interpreting  and  extrapolating  the  data  to  describe  the  seizure  distribution  would  be  ill- 
advised  as  it  is  almost  always  erroneous.24  Fitting  the  data  to  a  continuous  power  law 
function  is  illustrative  of  the  remarkably  robust  and  stable  nature  of  these  distributions 
(and  the  difficulties  in  evaluating  them). 

Figure  17  represents  the  same  data  set  using  logarithmic-scaled  axes  (log  of  the 
numbers  versus  the  log  of  the  quantities)  to  better  visualize  all  of  the  data.  The  data  are 
displayed  in  color-coded  bins,  with  heroin  containing  approximately  128  events/kg  per 
bin  in  red,  and  opium  containing  approximately  56  events/kg  per  bin  in  blue.  The  red  box 
shows  the  same  partial  set  of  the  data  that  was  displayed  in  Figure  16.  It  shows  a  very 
few  large  seizures  (one  seizure  of  10,000  kg  is  displayed  on  the  vertical-axis  as  0.0001), 
and  several  thousand  small  events  (of  1  kg  quantities.).  The  equations  shown  indicate 
similar  power  law  structures  (exponent  of  X"~  for  the  highest  power  in  each  denominator) 
for  both  opium  and  heroin  seizures  over  a  10  year  period  and  a  vast  range  of  data. 


This  concept  is  explained  in  detail  and  put  in  context  in  Appendix  A,  “Models  and  Simulations.” 

It  should  be  noted  that  the  Y  axis  in  Figure  18  (numbers  of  events  per  kilogram)  has  a  large  range  of 
eight  orders  of  magnitudes  (10s)  while  the  X  axis  (quantity)  has  a  range  of  four  orders  of  magnitude 
(104). 
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Source:  UNODC,  2011 

Figure  17.  UN  Analyzed  Heroin  and  Opium  Binned  Seizures  (2000-2009) 


This  representation  of  the  seizure  data  is  illustrative  and  consistent  with  the  equal 
risks  for  equal  returns  principle — that  the  data  represent  thousands  of  small  traffickers 
moving  small  loads  and  getting  smaller  rewards  while  there  are  only  a  few  traffickers 
moving  large  loads  with  potentially  high  rewards.  However,  ad  hoc  or  open  source 
reporting  often  (and  in  some  cases  intentionally  limited  reporting  to  limit  the 
documentation  burden)  under-represents  smaller  quantities  (1-100  kg)  by  many  fold 
which  distorts  this  distribution  (as  in  Figure  8).  As  a  result,  it  is  harder  to  extract  new 
information,  such  as  a  surrogate  for  price  or  other  valuable  information  derived  from 
seizures. 

In  response  to  IDA’s  presentation  of  these  emerging  techniques,  UN  researchers 
have  suggested  that  this  distribution  function  be  used  to  “fill  in”  the  missing  data  from 
limited  data  sets  to  make  better  estimates  from  seizures  of  actual  quantities.  They  concur 
that  smaller  quantities  are  likely  under-reported,  thus  anchoring  the  distribution  function 
at  the  high  end. 

IDA  finds  that  the  analysis  generated  by  the  CCDB-East  will  be  of  the  highest 
possible  quality  if  rigorous  data  validation  techniques  are  applied.  IDA  asked  statisticians 
in  the  UNODC’ s  Studies  and  Threat  Analysis  Section  within  the  Policy  Analysis  and 
Research  Branch  to  peer  review  this  technique  using  power  laws.  In  consultation  with 
CENTCOM,  the  analysts  agreed  on  the  utility  of  these  techniques  and  expressed  interest 
in  their  continued  application  to  SWA. 
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C.  What  Analytical  Capabilities  Are  Necessary? 

Because  of  the  illicit  nature  of  drug  trafficking  and  the  high-risk  operational 
environment  in  SWA,  it  is  not  realistic  to  expect  that  large  and  highly  detailed  sets  of 
drug  event  data  can  always  be  collected.  Nonetheless,  because  there  are  established 
quantitative  relationships  among  the  different  components  of  the  illicit  narcotics  system, 
data  it  is  not  feasible  to  collect  (e.g.,  opium  prices)  can  be  inferred  from  data  that  have 
been  collected  or  observed  (e.g.,  seizure  quantities  in  a  particular  time  period).  The 
accuracy  and  utility  of  these  techniques  for  generating  “surrogate”  data  have  been  tested 
and  validated  in  analyses  of  the  Western  Hemisphere  illicit  drug  markets.  Surrogate  data 
are  useful  when  they  are  extracted  from  systemically  collected  data  sets  that  contain 
actual  observations  (not  merely  averages  within  a  period)  and  that  sample  the  entire 
relevant  range  (e.g.,  high  volume  and  low  volume  shipments).  Current  USG  data  in  SWA 
do  not  meet  this  criterion. 

Figure  18  can  be  employed  to  illustrate  the  utility  of  surrogate  data.  The  IDA  study 
team  used  reports  of  opium  seizures  in  specific  periods  (months)  to  estimate  the  opium 
price  during  that  same  period.  The  blue  circles  represent  IDA’s  estimated  price — the 
surrogate  data  set.  The  IDA  team  then  fitted  a  curve  to  those  data  points,  generating  the 
dark  blue  line  in  the  figure.  The  red  line  records  the  average  dry  opium  price  reported  by 
the  UNODC.  The  UN’s  use  of  average  prices  masks  the  variability  in  dry  opium  prices, 
which  can  be  seen  in  the  price  estimates  IDA  generated  from  seizure  data.  The  UN  and 
the  IDA  lines  are,  however,  highly  correlated  with  nearly  identical  slopes.  The  surrogate 
data  (price  estimates  generated  from  seizure  reports)  could  be  further  validated  by 
comparison  to  other  characteristics  of  the  drug  market,  which  have  established  power-law 
mathematical  relationships  to  one  another. 
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Surrogate  Data  Correlates  With  Real-World  Data 
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Figure  18.  Surrogate  Opium  Price  versus  Real  Opium  Price 


D.  How  Can  Better  Event  Data  be  Used  to  Support  U.S.  Central 
Command’s  Programs? 

1.  Better  event  data  can  support  CENTCOM’s  programs  by  mitigating  some  of 
the  challenges  associated  with  data  limitations. 

As  demonstrated  in  the  previous  chapters,  the  major  shortcoming  associated  with 
SWA  opiate  data  is  that  it  is  extremely  limited  due  to  collection  efforts  that  are  neither 
methodical  nor  coordinated.  Recognizing  the  added  challenge  of  collecting  accurate  data 
on  an  inherently  secretive  and  illicit  market,  the  USG  must  rely  on  incomplete  data  sets 
for  its  analyses.  The  statistical  techniques  presented  in  this  document  offer  a  way  to 
mitigate  the  impact  of  these  data  gaps  by  correcting  erroneous  assumptions,  filling  in 
numbers  or  events  where  data  is  missing,  and  extracting  useful,  but  otherwise  unknown, 
information  from  limited  data  sets.  As  a  result,  CENTCOM  can  now  detennine,  with  far 
greater  accuracy,  the  actual  scope  of  the  heroin  and  opium  trades  in  its  and  neighboring 
AORs.  With  more  accurate  data,  CENTCOM  can  pinpoint  with  better  precision  when 
certain  government  policies  and  interventions  actually  begin  to  work. 

For  example,  throughout  the  course  of  this  study  various  interagency  stakeholders 
identified  specific  data  gaps  where  additional  information  could  be  extremely  helpful. 
With  additional  research  and  testing,  the  techniques  presented  in  this  document  can  likely 
be  used  to  provide  insight  into  the  following  aspects  of  the  opium  and  heroin  trades: 
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a.  Existence  of  stockpiles  and  the  effect  on  prices 

Based  on  the  proven  power  law  distributions  of  drug  data,  IDA  researchers 
hypothesize  that  the  sizes  of  opium  stockpiles  are  overestimated.  Currently  existing 
stockpile  estimates  are  calculated  by  subtracting  missing  amounts  from  the  entire  amount 

'If, 

of  opiates.  However,  it  is  likely  that  many  analysts  used  averaged  samples  to  calculate 
the  missing  amounts.  As  demonstrated  in  this  document,  using  averages  rather  than 
medians  has  the  effect  of  significantly  inflating  the  value  being  measured.  In  this  case, 
the  following  could  be  overestimated:  production  (cultivation),  seizures,  or  consumption 
rates.  Ongoing  price  fluctuations  in  Afghan,  Asian,  and  world  markets  for  opium  appear 
to  confirm  the  finding  that  there  are  not  large  stockpiles  of  opium  available  to  cushion 
price  movements,  and  that  there  likely  are  not  large  or  coordinated  trafficking 
organizations  that  would  draw  on  any  existing  stockpiles. 

b.  Precursor  chemicals 

It  may  be  possible  to  extract  infonnation  about  drug  purity  from  limited  drug  event 
data  collected  in  SWA  using  the  analytic  methods  presented  here.  Although  Afghan 
personnel  and  DEA  personnel  in  Afghanistan  collect  some  information  about  drug  purity, 
limited  drug  testing  infrastructure  and  ongoing  security  issues  make  this  data  rare.  Data 
on  drug  purity  could  indirectly  provide  information  about  the  supply  of  precursor 
chemicals  reaching  Afghanistan  and  what  affect  counter-measures  may  have  on 
availability. 

c.  Locations  of  drug  labs 

It  is  plausible  that  tracking  price-like  surrogate  data  extracted  from  limited  seizure 
data  could  contribute  to  a  better  understanding  of  the  location  of  drug  labs.  This  is  based 
on  the  assumption  that  the  price  changes  as  drugs  move  from  production  labs  in  SWA  to 
their  final  destination  in  European  markets.  The  surrogate  data  could  not  support  a 
capacity  to  precisely  target  specific  locations,  but  it  could  provide  insight  into  the  amount 
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of  production  in  a  given  region  or  where,  in  a  general  sense,  numerous  labs  might  exist.' 


The  UNODC  estimates  missing  amounts  by  comparing  flow  volumes  with  consumption  rates.  The 
difference  indicates  the  existence  of  stockpiles. 

Incidentally,  the  effectiveness  of  attacking  drug  labs  depends  on  the  threshold  of  effects,  i.e., 
destroying  enough  labs  to  make  an  impact.  Two  IDA  papers  describe  deterrence  in  detail:  Barry  D. 
Crane,  Deterrence  Effects  of  Operation  Frontier  Shield ,  IDA  Paper  P-3460  (Alexandria,  VA:  Institute 
for  Defense  Analyses,  March  1999)  and  Robert  W.  Anthony,  Barry  D.  Crane,  Stephen  F.  Hanson, 
Deterrence  Effects  of  Peru ’s  Force-down/Shoot-down  Policy ,  IDA  Paper  P-3472,  (Alexandria,  VA: 
Institute  for  Defense  Analyses,  April,  2000)  Using  the  methods  presented  in  this  document,  policy 
makers  could  plan  and  assess  a  campaign  to  eliminate  drug  labs  in  a  given  region,  versus  the  current 
operational  trial  and  error  method. 
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d.  Quantities  and  physical  locations  of  seizures 

Traditional  methods  of  data  analysis  have  not  accounted  for  the  inherent  tendency  to 
overestimate  quantities  extrapolated  from  small  data  samples.  There  is  a  wide  difference 
between  an  average,  or  mean,  value  for  seizure  size,  and  the  median.  The  methods 
presented  here  can  provide  more  accurate  analyses  of  drug  quantities  and  can  extract 
more  value  from  drug  data  to  characterize  individual  data  sets  and  trends  exhibited  by 
time-sequenced  data  collections. 

e.  Intermediate  drug  prices  within  Afghanistan 

The  analytic  methods  used  to  extract  additional  infonnation  from  drug  data  lead  to 
better  descriptions  of  the  drug  market  than  traditional  methods.  For  example,  this 
document  demonstrates  that  Afghan  drug  prices  appear  correlated  to  price-like  surrogate 
data  for  drugs  in  SWA.  Better  knowledge  of  intennediate  prices  can  help  analysts  map 
and  explain  trafficking  behavior  in  the  Afghan  market  and  evaluate  the  effectiveness  of 
CN  efforts. 

2.  CENTCOM  can  improve  its  knowledge  of  its  adversaries,  how  they  resource 

themselves,  and  how  they  operate  based  on  perceived  risk. 

More  fundamentally,  IDA’s  research  has  suggested  a  fundamental  finding  regarding 
criminals’  tolerance  for  risk-taking.  Past  IDA  research  demonstrated  that  drug  traffickers, 
insurgents,  and  terrorists  form  similar  organizational  structures — loosely-connected  webs 
of  small,  specialized  cells,  etc.'  Illicit  networks  form  organizational  structures  that  allow 
them  to  operate  asymmetrically  and  clandestinely  against  enemies  of  overwhelming  size 
and  resources.  Key  to  the  illicit  network  structure  is  the  number  of  so-called  “trusted 
agents”  employed  by  the  organization.  If  the  organization  employs  too  many,  it  is 
vulnerable  to  infiltration  and  enemy  action.  If  it  employs  too  few,  operations  will  be 
difficult  or  impossible  to  execute. 

IDA  research  indicates  that  most  components  of  illicit  drug  networks  assume  risk  at 
a  similar  rate.  Similarly,  IDA  observed  that  this  relationship  holds  for  all  levels  of  a  drug 
market,  from  wholesaler  to  street-level  dealer  and  among  different  geographic  regions  in 
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the  world."  Illicit  trafficking  organizations  will  take  small  risks  for  small  returns  because 
the  perceived  risk  is  low.  Only  a  few  take  very  big  risks  for  large  returns,  because  the 
perceived  risk  is  very  high.  This  risk-taking  behavior  is  otherwise  known  as  equal  risks 
for  equal  returns.  For  example,  small  seizures  of  less  than  1  kg  amounts  are  far  more 
frequent  than  seizures  of  a  ton.  Similarly,  more  traffickers  traffic  smaller  amounts  more 


R.  Anthony  and  A.  Fries,  “Empirical  Modelling  of  Narcotics  Trafficking  from  Farm  Gate  to  Street,” 
UNODC  Bulletin  on  Narcotics:  Illicit  Drug  Markets  LVI,  nos.  1  and  2  (2004):  7-8. 

Ibid.,  16. 
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frequently  than  very  large  amounts.  Likewise,  more  users  buy  smaller  amounts  more 
often  than  users  who  buy  large  amounts.  In  other  words,  the  rate  of  criminal  behavior 
(i.e.,  the  interval  between  events)  is  faster  for  smaller  risks  and  slower  for  larger  risks. 

This  distribution  of  rates  appears  to  be  a  fundamental  signature  of  irregular  gangs, 
insurgents,  traffickers,  and  any  other  illicit  organizations  avoiding  detection  by  a  state. 
Because  these  rates  are  linked  across  networks,  analysts  are  able  to  construct  drug  market 
indices  from  the  data.  Time  intervals  between  drug  trafficking  events  (e.g.,  two-month 
intervals  between  one  ton  shipments)  are  observed  to  be  a  stable  analytic  indicator. 
Changes  in  the  rate  of  trafficking  incidents  are  consistent  with  changing  levels  of  market 
activity.  There  can  be  many  reasons  for  changes  in  market  activity,  such  as  bad  crop 
yields,  supply-chain  disruptions,  or  increased  LE  seizures  on  the  supply  side  or  decreased 
drug  consumption  on  the  demand  side.  To  resolve  causal  ambiguities,  additional  (demand 
side)  data,  such  as  positive  drug  testing  rates,  should  be  used  to  corroborate  hypotheses 
developed  from  seizure  data.  In  the  underlying  analytical  products  supporting  formal  UN 
publications,  this  process  has  been  used  repeatedly  for  the  last  ten  years,  but  not  in  USG 
products  since  2004. 

The  research  presented  in  this  document  suggests  that  this  risk-taking  behavior  is 
ubiquitous  and  applies  not  only  to  global  criminal  activities  but  also  to  insurgent  forces — 
many  of  which  are  funded  through  the  drug  trade.  This  fact  can  assist  CENTCOM  in 
designing  and  evaluating  counterinsurgency  (COIN)  strategies.  Moreover,  the  methods 
presented  here  will  foster  greater  efficiencies  by  making  it  possible  for  CENTCOM  to: 
(1)  improve  the  accuracy  of  drug  data  without  expanding  existing  programs,  (2)  shorten 
the  reaction  time  of  U.S.  government  entities  to  changes  in  a  drug  market,  and  (3) 
measure  and  analyze  the  effects  of  CN  programs. 
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3.  Recommendations 


1.  Entries  in  the  CCDB-East  should  be  in  the  distribution  of  raw  event  data,  not 
averaged  estimates. 

In  order  to  apply  the  analytic  methods  described  in  this  study,  analysts  must  have 
access  to  raw,  event-level  data  sets.  Averaged  values  for  any  measurements  of  drug 
activity  over  a  period  of  time  (for  example,  average  seizure  size,  average  number  of 
seizures  made,  average  amount  of  drugs  used  in  the  past  month,  etc.)  do  not  have  the 
degree  of  granularity  needed  to  model  the  drug  market.  The  process  of  calculating  an 
average  value  for  a  set  of  data  hides  critical  infonnation  needed  for  understanding  the 
system.  It  is  important  to  also  evaluate  the  median  and  compare  it  to  the  average,  and 
even  more  important  to  examine  the  actual  distribution  of  data. 

2.  Analysis  from  data  contained  in  the  CCDB-East  should  maintain  the  integrity 
of  separate  data  sets. 

While  consolidated  data  sets  may  list  the  total  number  of  events,  they  cannot  show 
patterns  in  the  seizure  data  or  support  detailed  analysis  of  events.  This  is  true  because 
data  sources  that  are  combined  into  a  single  data  set  will  distort  the  time-dependent 
information  contained  in  each  data  point.  There  will  invariably  be  differences  in  the 
collection  methodologies  of  the  two  distinct  data  sets  and  losing  critical  time-dependent 
infonnation  will  make  relative  measurements  difficult.  For  this  reason,  it  is  critical  that 
the  CCDB-East  and  different  data  sources  not  be  combined  into  one  data  set.  If  they  are, 
then  it  must  be  possible  to  differentiate  between  each  data  source  and  compare  and 
analyze  them  separately.  Comparing  different  data  sets  allows  analysts  to  see  if  they 
exhibit  the  same  patterns  using  time  series  and  non-CCDB  data  validation  sets. 

3.  Formal  coordination  of  the  data  collection  processes  should  be  implemented 
among  the  interagency  and  multinational  partners. 

Voluntary  data  collection  and  information  sharing  should  be  encouraged  at  all 
levels.  An  example  would  be  making  collection  tools  (e.g.,  drug  testing  equipment) 
available  to  personnel  in  return  for  data  submissions.  Collection  procedures  should  be 
efficient  and  easy  to  follow.  Care  must  be  taken  to  ensure  that  interagency  partners  are 
not  required  to  repeatedly  enter  the  same  data  into  multiple  databases,  which  discourages 
voluntary  compliance.  Fonnal  coordination  of  data  collection  among  the  interagency 
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partners  will  facilitate  identifying  data  requirements,  data  sharing,  and  the  systematic  use 
of  analytic  methods  to  improve  data  quality,  which  improves  efficiency  and  reduces 
costs. 

4.  CCDB-East  should  have  a  system  for  assigning  and  vetting  event  data 
confidence  levels. 

As  the  CCDB-East  evolves,  it  will  become  critical  to  ensure  that  a  process  similar  to 
the  one  that  exists  for  the  current  CCDB,  is  in  place  for  assigning  and  vetting  confidence 
levels  for  each  entry  into  the  database.  Considering  the  sporadic  nature  of  data  collection 
in  SWA,  where  the  vast  majority  of  reports  will  likely  be  rated  a  “P”  with  only  a  few  “C” 
and  “S”  reports,  this  function  will  be  even  more  important. 

5.  Data  in  the  CCDB-East  should  be  compared  with  independent  data  sets  that 
provide  some  validation  potential. 

Sufficient  independent  event  data  of  high  enough  quality  to  generate  dependable 
analyses  and  corroboration  is  not  currently  being  collected  by  the  USG,  nor  is  it  easy  to 
collect,  in  SWA.  Although  additional,  independent  data  to  validate  CCDB-East  data  are 
sparse,  some  data  collected  by  the  UNODC,  such  as  consumption  statistics,  can  be  used 
for  this  purpose.  As  a  result,  IDA  finds  that  data  from  the  CCDB-East  should  be 
compared  with  additional  independent  data  sources  (such  as  price  and  consumption, 
which  the  UNODC  World  Drug  Report  attempts  to  assimilate)  as  well  as  any  other  event 
databases  in  the  region,  to  provide  context  for  the  event  data  and  some  validation 
potential.  Where  validation  with  independent  data  is  impossible  or  insufficient,  then 
validation  using  power  laws  may  be  used  to  test  newly  acquired  data.  This  technique  is 
described  in  the  next  recommendation. 

6.  Administrators  of  the  CCDB-East  should  ensure  the  thorough  technical 
validation  of  all  event-level  data  to  maximize  the  utility  of  limited  data  sets. 

Event  data  contained  in  the  CCDB-East  must  be  validated  for  accuracy.  This 
validation  process  is  different  from  the  existing  process,  which  validates  for  consistency 
across  event  reporting,  even  when  events  may  not  be  confirmed  or  substantiated.  There 
are  several  advanced  analytical  methods  that  can  be  applied  to  event  data  to  assess  its 
quality,  including  analysis  of  comparable  and  independent  time  series  data  sets  to  see 
whether  patterns  exist  (therefore  increasing  confidence  in  the  data’s  quality)  or 
comparison  with  independent  data  sources,  such  as  price  and  purity,  to  see  whether  they 
“tell  the  same  story.”  Power  law  distributions  can  serve  as  valuable  tools  to  validate  new 


Purity  data  would  ideally  provide  an  additional  data  source  for  validation  purposes,  but  currently  very 
little  is  known  about  the  purity  levels  of  SWA  opiates.  There  are,  however,  proposals  for  the  UN  to 
fund  several  laboratories  in  the  future  to  measure  drug  quality  and  purity. 
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and  existing  repositories  of  drug  data,  thus  improving  the  quality  of  the  data  used  to 
extract  infonnation.  Because  low-confidence  data,  including  open-source  data,  are 
problematic,  the  use  of  analytic  methods  to  correct  for  inaccuracies  or  bias  can  improve 
the  overall  understanding  of  the  drug  system.  Power  laws  provide  a  way  to  independently 
check  for  missing  data,  new  market  conditions,  or  changes  in  collection  procedures 
(when  collection  procedures  are  changed  without  recalibrating  the  data  analysis,  the  data 
can  no  longer  provide  consistent  relative  results).  Preliminary  testing  of  the  new 
statistical  techniques  presented  here  demonstrate  a  proof  of  principle,  but  additional 
testing  is  needed  to  standardize  the  methodology  and  make  it  fully  operational.  That  said, 
the  use  of  power  laws  over  the  last  15  years  has  resulted  in  more  accurate  estimates  of 
trafficking  conditions  and  the  detection  of  hidden  trafficking  patterns.  It  has  also  allowed 
policy  makers  to  assess  the  impact  of  government  interventions,  such  as  the  demise  of  air 
and  surface  trafficking  conveyances  in  the  WH  and  the  impact  of  the  surge  in  Iraq. 

7.  CCDB-East  administrators  and  data  collectors  should  agree  on  minimal 

reporting  thresholds  that  accurately  sample  the  universe  of  drug  trafficking 

events. 

CCDB-East  administrators  and  data  collectors  should  agree  on  minimal  reporting 
thresholds  that  accurately  sample  the  universe  of  drug  trafficking  events.  For  example,  if 
the  majority  of  seizures  of  a  given  drug  are  in  the  range  of  one  and  five  kilograms, 
reporting  thresholds  must  be  low  enough  to  capture  this  infonnation.  Limiting  data 
collection  to  seizures  of  hundreds  of  kilograms  prevents  the  extraction  of  more  detailed 
information. 

8.  CENTCOM  should  extract  surrogate  data  where  incomplete  data  sets  exist. 

Extracting  and  using  surrogate  or  inferred  data — calculated  from  directly  observable 
data  and  used  where  the  data  desired  for  analysis  does  not  exist — in  lieu  of  incomplete 
data  sets  is  a  valuable  method  to  make  better  estimates  of  the  drug  system.  Moreover,  this 
analytical  technique  can  reveal  previously  unknown  infonnation.  Although  this  analytical 
method  works  for  systematically-collected  UN  data,  surrogate  data  cannot  be  derived 
from  the  chance  data  that  is  cunently  contained  in  the  CCDB-East.  As  such,  the  CCDB- 
East  should  exploit  UN  data  to  the  fullest  extent  while  systematic  data  collection 
procedures  are  established. 
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4.  Conclusions 


During  the  course  of  this  research,  IDA  examined  issues  related  to  data  availability, 
collection,  and  analysis  for  opiates  in  SWA.  Researchers  first  studied  the  CCDB  and 
observed  the  associated  process  for  evaluating  seizure  events  in  the  WH.  This  provided 
the  context  for  understanding  USG  drug  data  collection  efforts  in  SWA.  Next,  technical 
experts  examined  drug  event  databases  shared  by  the  UNODC  and  the  DIA  to  leam  how 
analyses  of  drug  data  can  be  improved,  particularly  in  SWA,  especially  in  Afghanistan 
where  there  is  limited  data  collection  capacity. 

A  critical  observation  from  this  study  is  the  starkly  different  operating  environment 
for  CN  programs  in  SWA  when  compared  to  those  in  the  WH.  U.S.  forces  do  not  have 
the  Detection  and  Monitoring  (D&M)  assets,  capabilities  or  collection  platforms  in  SWA 
that  have  accrued  over  decades  in  the  WH.  A  more  fundamental  challenge  is  that  very 
little  reliable  information  about  drug  seizures  is  available.  The  U.S.  military  and  ISAF 
personnel  currently  operating  in  Afghanistan  have  limited  involvement  in  CN 
operations — mainly  providing  security  to  Afghan  and  U.S.  law  enforcement  personnel 
who  perform  CN  functions.  Although  the  DEA  and  vetted  Afghan  units  are  actively 
performing  CN  missions,  their  ability  to  carry  out  robust  and  thorough  data  collection  is 
constrained  by  limited  resources  (personnel  and  equipment)  and  even  more  by  the  high- 
risk  security  environment  in  Afghanistan. 

As  a  result,  the  majority  of  event  data  reported  by  the  U.S.  military  and  allied 
partners  originates  from  chance  seizures.  These  data  are  used  to  generate  estimates  of 
regional  drug  trafficking  activity.  IDA  research  suggests  that  the  standard  interpretation 
of  USG  data  significantly  overestimates  the  threat.  Moreover,  the  use  of  averages  (rather 
than  medians)  creates  large  errors  that  make  it  difficult  to  assess  the  success  of  USG 
interventions.  One  way  to  mitigate  this  challenge  is  to  place  more  emphasis  on  opium 
data  (versus  heroin  data)  which  is  proven  to  be  a  better  indicator  due  to  its  greater 
sensitivity  to  enforcement  actions.  In  addition,  the  USG  could  benefit  greatly  from 
analytic  methods  that  add  value  to  existing  drug  data.  As  troop  levels  draw  down  and 
these  reporting  streams  decline,  these  analytic  methods  will  become  even  more  important 


As  stated  above,  chance  drug  seizures  refer  to  drug  seizures  made  during  a  military  operation  that  was 
not  planned  or  intended  to  be  a  CN  operation. 

See  Pietschman,  “Price-setting  Behaviour,”  105. 
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to  understand  and  implement.  The  ability  to  extract  valuable  information  from  limited 
sets  of  drug  data  without  assigning  additional  personnel  or  other  assets  to  create  more 
robust  data  collection  efforts  is  critically  important  in  the  context  of  the  resource- 
constrained  operating  environment  in  SWA.  Streamlining  CN  operations  could  develop 
more  efficient  strategies  and  programs  that  can  be  effectively  measured  and  analyzed. 

This  task  was  built  on  many  of  the  foundational  principles  provided  by  decades  of 
previous  IDA  research  focused  on  the  cocaine  market  in  the  WH,  specifically  the 
consistent  patterns  of  drug  trafficking  behavior  that  result  in  power  law  distributions.  A 
power  law  distribution  of  drug  event  data  can  be  observed  for  numerous  characteristics  of 
a  drug  market,  including  trafficker  behavior  (equal  risks  for  equal  returns),  size  of  drug 
seizures,  and  quantity  of  drugs  used,  among  other  things.  But  until  now  researchers  had 
not  yet  explored  the  applicability  of  this  pattern  to  opiate  data  in  SWA. 

Most  importantly,  IDA  has  developed  techniques  for  extracting  valuable 
infonnation  from  limited  data  sets,  such  as  estimates  of  price  data  where  actual  data  does 
not  exist.  This  value  is  known  as  a  price-like  surrogate  and  when  compared  to  real-world 
opium  price  data  collected  by  the  UN,  IDA  observed  consistency  in  overall  trends,  thus 
validating  the  use  of  the  surrogate.  These  analytic  methods  are  evolving  techniques  and 
will  benefit  from  additional  testing,  but  initial  results  seem  to  indicate  that  a 
systematically  collected  set  of  seizure  data  can,  in  some  circumstances,  be  used  to  derive 
a  price-like  surrogate  value  for  drug  prices — a  highly  valuable  piece  of  infonnation  for 
measuring  drug  market  trends  in  Afghanistan.  For  example,  a  price-like  surrogate  may 
help  analysts  identify  regions  where  drug  supply  seems  high  (and  price  low),  potentially 
indicating  the  presence  of  drug  labs  or  possibly  stockpiles.  Analysts  may  also  use  the  data 
to  discern  previously  unknown  trafficking  routes,  where  drug  prices  rise  as  drugs  are 
passed  from  buyers  to  sellers  from  the  source  zone  to  consumer  markets.  A  similar 
approach  (requiring  additional  study  and  testing)  may  enable  analysts  to  extract 
infonnation  about  drug  purity  from  event  data  (test  results  using  cocaine  data  suggest  this 
is  possible)  and  thus  derive  knowledge  about  the  availability  of  precursor  chemicals. 

In  order  to  maximize  the  utility  of  CN  data,  the  USG  must  adopt  the  following 
measures: 

•  Use  raw  (not  averaged)  data  sets  and  keep  data  sources  separate 

-  Construct  distributions  based  on  events  per  kilogram  versus  summarized 
quantities 

-  Calculate  rates  of  trafficking  by  parsing  the  larger  distribution  into  smaller 
desired  time  blocks 


See  Appendix  B,  “Analysis  of  the  Cocaine  Market.” 
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•  Employ  systematized  data  collection  methods  and  ensure  data  is  collected  across 
the  spectrum  of  event  sizes  (small  as  well  as  large  events) 

-  Compare  USG  with  UN  data  (or  data  proven  to  be  consistent  with  the 
underlying  functional  power  law  distribution)  to  detennine  an  estimate  of 
measurement  error 

-  Use  the  comparison  to  provide  a  basis  for  estimating  corrections  for  the 
USG  data 

-  Develop  a  correction  protocol  for  USG  data  that  overcomes  the  bias 
associated  with  under-sampling  small  seizures  missing  from  chance  data 

•  Categorize  event  data  by  confidence  levels  and  validate  it  using  time  series 
plots,  comparison  with  independent  variables,  and  underlying  power  law 
functional  distributions 

If  adopted,  these  new  analytic  methods  being  developed  by  IDA  researchers  could 
provide  cost  savings  to  Department  of  Defense  (DOD)  by  making  it  possible  to  extract 
otherwise  unavailable  but  valuable  infonnation  from  existing  data  and  to  validate 
available  data  sets.  When  the  analytic  method  becomes  fully  operational,  it  will  be 
possible  to  “do  more  with  less”  without  committing  additional  resources  to  data 
collection  efforts. 
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Appendix  A 
Models  and  Simulations 


Terrorist  and  Criminal  Behaviors  Learned  from  Countering  Drug 
Systems 

Since  the  mid-1990s  the  Institute  for  Defense  Analyses  (IDA)  has  conducted 
research  on  irregular  adversaries:  criminal  drug  enterprises  (producers,  cultivators, 
traffickers,  dealers,  and  users)  and,  more  recently,  insurgents,  terrorists,  and  their 
sympathizers.  The  basic  characteristic  of  these  organizations  is  that  they  are  small  groups 
engaging  a  larger  and  more  powerful  state  or  group  that  is  attempting  to  eliminate  their 
illicit  behaviors  through  enforcement  actions.  IDA  research  has  focused  on  advanced 
statistical  methods  to  quantify  and  ultimately  test  these  behaviors,  and  has  found  that 
consistent  patterns  of  behavior  characterized  by  power  laws  can  allow  analysts  to 
estimate  the  consequences  of  enforcement  actions  and  other  aspects  of  the  drug  system 
(to  include  supply,  demand,  consumption,  and  volumes  of  drugs  trafficked,  among 
others). 

One  of  the  fundamental  principles  of  these  analyses  is  that  the  risk  traffickers  and 
criminals  are  willing  to  take  is  proportional  to  their  expected  return.  This  is  known  as  the 
equal  risk  for  equal  returns  principle.  1  Moreover,  if  these  groups  try  to  organize  into  a 
larger  organization,  then  the  capture  of  one  individual  may  lead  to  an  action  against  all.  If 
the  group  is  too  small,  then  it  is  not  very  competitive  with  other  similar  organizations.  As 
a  result,  the  size  of  a  criminal  organization  most  often  depends  on  the  severity  of  the 
enforcement  penalty,  which  is,  in  turn,  a  factor  of  the  commodity  type  (the  more 

dangerous  and  valuable  the  commodity,  the  greater  the  risk  in  trafficking  that 

2 

commodity)."  A  second  observation  regarding  irregular  organizations  is  that  size  of  an 
organization  appears  to  be  inversely  proportional  to  the  risks  taken  and  effective  actions 

■j 

taken  against  the  organization  depend  on  risk  dependent  thresholds.  ~  Below  the  threshold 


These  concepts  and  observations  were  first  made  to  the  UNODC  on  Afghanistan  as  UN  personnel  left 
Afghanistan  after  the  attacks  of  September  11th,  2001:  A.  Rex  Rivolo,  “Structural  Dynamics  of  the 
Cocaine  Market,”  IDA  Joint  Meeting  with  UNODC,  September  20,  2001,  Vienna. 

Thomas  Pietschmann,  “Price-setting  Behaviour  in  the  Heroin  Market,”  UNODC  Bulletin  on  Narcotics: 
Illicit  Drug  Markets  LV1,  nos.  1  and  2  (2004):  106. 

Risk  dependent  thresholds  were  observed  when  the  small  adversary  units  parse  risk,  the  more  severe 
the  risk,  the  lower  the  threshold  and  the  smaller  the  organizational  core.  Thresholds  were  determined 
from  repeated  observations  of  criminal,  trafficking,  and  combat  activities. 
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the  organization  slowly  improves  its  ability  to  evade  detection,  above  the  threshold  there 
is  rapid  decline.  This  behavior  was  also  observed  to  be  true  for  insurgent  networks 
(where  the  combat  insurgent  groups  were  relatively  small  because  the  kinetic  penalty  is 
severe)  and  for  their  larger  support  networks  (where  the  enforcement  penalty  for  arrest 
and  criminal  conviction  is  less  severe).4  Finally,  it  was  observed  that  the  organizational 
structures  developed  to  mitigate  risk  were  almost  universal5  and  ubiquitous  in  nature. 

This  equal  risks  for  equal  returns  principle  means  that  power  laws — i.e.,  that  more 
criminals  take  smaller  risk  for  smaller  return  while  a  few  take  larger  risks  for  larger 
returns — exist  in  many  aspects  of  the  drug  trade  and  insurgent  behavior.  This  analytical 
observation  has  many  practical  applications  that  are  critical  for  policy  makers  and 
operational  commanders:  revising  drug  stockpile  estimates  to  realistic  levels,  calculating 
more  accurate  flow  and  use  estimates,  determining  successful  operational  outcomes,  and 
most  importantly,  making  terrorist  denial  and  deception  tactics  to  avoid  detection  far 
more  difficult.  In  particular,  when  irregular  forces  and  drug  traffickers  are  deterred  the 
effect  has  been  observed  to  be  inversely  proportional  to  the  intervention  technique. 

The  mass  distribution  of  an  illicit  substance  among  market  participants  is  observed 
to  follow  a  power  law-like  distribution  with  smaller  events  occurring  more  frequently 
than  larger  events.6  In  licit  economies  distributions  of  wealth  obey  similar  power  laws, 
such  as  the  median  prices  of  homes.7  Correspondingly,  indicators  of  wealth  in  both 
research  and  popular  publications  have  exclusively  relied  on  percentiles  such  the  fiftieth 
percentile,  also  known  as  median,  or  upper  and  lower  first,  fifth  or  tenth,  etc.  as  wealth 
distribution  descriptors,  as  opposed  to  the  mean  or  average.  It  is  understood  that  the  mean 
has  little  descriptive  value  for  these  situations  because  the  means  of  most  power  law 
distributions  do  not  converge,  even  with  very  large  sample  sizes.  As  a  consequence,  if 
different  subsets  of  a  power  law-distributed  population  are  selected  (e.g.,  monthly 
compilations),  the  calculated  sample  mean  for  each  subset  will  likely  differ  greatly. 


Observed  and  demonstrated  by  A.  Rex  Rivolo,  as  a  member  of  a  special  advisory  team  to  the  Third 
U.S.  Corps,  Iraq,  during  surge  operations  after  reviewing  published  papers  by  Professor  Neil  F. 
Johnson. 

A.  Rex  Rivolo,  “Structural  Dynamics  of  the  Cocaine  Market,”  IDA  Joint  Meeting  with  UNODC, 
September  20,  2001,  Vienna. 

Cauchy-like  is  a  more  technically  accurate  description  of  this  distribution,  but  for  understandability  the 
more  commonly  understood  power  law  analytic  distribution  is  used. 

This  feature  is  also  characterized  as  following  a  Pareto  power  law  distribution. 

The  mean  is  non-convergent  for  power  law  distributions  with  a  power  law  exponent  (shape  parameter) 
of  less  than  two.  For  shape  parameter  values  between  two  and  three,  although  theoretically  convergent, 
it  does  so  very  slowly.  A  large  number  of  sample  points  are  required  to  achieve  even  modest 
convergence. 
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This  appendix  examines  the  analytical  difficulties  of  interpreting  the  power  law-like 
data  that  underpins  the  behavior  of  drug  traffickers,  terrorists,  and  insurgents  in  order  to 
process  it  into  valuable  information.  Estimating  informative  quantities  from  these 
distributions  requires  more  than  using  simple  averages  and  sums.  Sampling  small 
portions  of  the  data  may  produce  randomly  varying  averages  that  are  very  poor 
estimators.  Averages,  small  summed  quantities,  and  other  important  attributes  cannot  be 
simply  computed  without  recognizing  the  extreme  nature  of  these  asymmetric 
distributions.  Averages  may  be  many  times  greater  than  medians,  which  are  convergent 
statistics.  Small  samples  of  asymmetric  distributions  will  produce  highly  variable 
estimates  that  tend  to  skew  reported  results  in  favor  of  the  traffickers,  insurgents,  or 
criminals. 

Figure  A-l  shows  the  functional  distributions  of  systematically  and  carefully 
collected  United  Nations  (UN)  opium  and  heroin  data  binned  into  equal-sized  collections 
of  100  data  points.  The  presentation  was  sized  to  include  both  the  averages  and  medians, 
but  not  all  the  data  was  included  since  many  larger  values  are  further  out  on  the 
horizontal  axis.  Averages  were  repeatedly  observed  in  many  cases  to  be  very  poor 
representations  of  useful  infonnation  from  the  data  because  the  averages  are  heavily 
influenced  by  a  very  few  large  seizure  events.  Medians  are  more  accurate  since  these 
values  represent  the  fiftieth  percentile  and  do  not  overestimate  the  typical  quantities 
seized. 
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x  variable  for  both  heroin  and  opium 
Seizure  normalization  constants  are: 

Cheron  —  3,500  and  Cppuj,,,  —  1,000,  Atxjth  —  2 
Bheron=5and  BopKJT1  =  50 

#=events/kg*quantity  at  1  mt  yields  -20 
heroin  and  ~50  opium  seizure  events 
Averages  are  very  poor  seizure  measures 

Heroin  average  =17;  median  =  1 .2  kg 
Opium  average  =  99,  median  =  11.5  kg 


Quantity  Seized  (kg) 


Source:  UNODC 

Figure  A-1.  Number  of  Seizure  Events  per  kg  of  Opium  and  Heroin 
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The  opium  and  heroin  data  generate  very  robust  and  stable  power  law  models 
(distribution  or  functional  shape  distribution  x'  ).  Knowing  the  stable  model  distributions 
allows  extraction  of  more  valuable  infonnation  than  monthly  averages  that  would  be 
better  described  by  the  distribution  function.  This  is  an  important  use  of  the  data  because 
it  allows  analysts  to  accomplish  several  objectives.  First,  they  can  validate  and  test  newly 
acquired  data.  A  new  data  source,  such  as  a  new  UN  reporting  country,  can  reveal 
previously  hidden  patterns  of  trafficking  or  reveal  changes  in  risk-taking  and  drug 
trafficking  organizational  operations.  Secondly,  analysts  can  test  the  effectiveness  of 
operations  with  updated  data.  Trafficking  that  is  declining  very  slowly  is  observed9  to  be 
below  the  threshold  in  seizures,  arrests,  or  kinetic  interventions,  while  rapid  drops  in 
seizures  suggest  that  an  above  the  threshold,  efficient,  and  effective  enforcement 
operation  may  be  taking  place.  Moreover,  operations  can  be  tested  on  a  small-scale  and  at 
reasonable  costs  to  determine  whether  a  policy  or  tactic  works.  Lastly,  analysts  can 
improve  their  ability  to  measure  the  success  of  operations.  Monthly  averages  will  tend  to 
exaggerate  enemy  or  trafficking  performance.  Once  more  accurate  estimates  are  made, 
policy  makers  can  measure  enemy  performance  with  greater  accuracy.  These  models 
guided  Western  Hemisphere  operations  to  defeat  cocaine  trafficking  by  reducing  user 
testing  rates  more  than  80%  and  supply  availability  by  75%. 

Figure  A-2  compares  the  UN  data  collected  over  ten  years  ending  in  2010  with  the 
United  States  Government  (USG)  data  on  heroin  and  opium.  It  illustrates  the  significant 
difference  between  the  UN’s  opium10  event  data  (2,606  total  events)  and  the  USG’s 
chance  data  (295  total  events).  Chance  data  refers  to  drug  seizures  made  during  a  military 
operation  that  was  not  planned  or  intended  to  be  a  CN  operation.  Due  to  limited 
collection  assets,  capabilities,  or  systematic  data  collection  procedures  and  only  three 
years  of  collection  ending  in  2010,  most  USG  event  data  falls  into  this  category. 
Logarithmic-scaled  axes  are  used  (log  of  the  events  and  the  log  of  the  quantities)  to  show 
the  extraordinary  range  of  all  data  quantities  and  the  stability  of  the  seizure  distributions. 
(It  should  be  noted  that  the  log-log  display  enables  the  display  across  an  enormous  value 
range  while  continuing  to  make  visible  the  concentration  of  data  at  one  end  of  the  scale.) 


Barry  D.  Crane,  Deterrence  Effects  of  Operation  Frontier  Shield,  IDA  Paper  P-3460  (Alexandria  VA: 
Institute  for  Defense  Analyses,  1997). 

Opium  data  was  used  over  heroin  data  due  to  its  greater  sensitivity  to  enforcement  actions.  See 
Pietschman,  “Price-setting  Behaviour,”  105. 
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Figure  A-2.  A  Comparison  of  UN  Opium  and  USG  Event  Data 

Different  distributions  stemming  from  different  collection  methodologies  have 
critical  implications  for  the  calculated  averages  and  medians,  and  result  in  a  sampling 
bias.  Because  UN  data  is  collected  in  a  systematic  fashion,  a  proportionate  number  of 
events  are  collected  at  both  the  low  and  high  ends  of  the  distribution  curve.  In  contrast, 
USG  chance  data  significantly  under  samples  the  low  end  of  the  distribution  curve  (small 
events).  This  is  why  the  two  curves  are  consistent  with  each  other  for  seizures  greater 
than  100  kg  but  there  is  significant  deviation  for  events  less  than  100  kg  (see  the  box  in 
Figure  A-2).  In  other  words,  the  UN  data  consistently  shows  two  times  as  many  events 
recorded  for  events  larger  than  100  kg  while  it  has  approximately  230  times  as  many 
events  recorded  for  1  kg  events,  (see  call-out  boxes  in  Figure  A-2). 

Figure  A-3  shows  how  the  known  power  law  distribution  derived  from  the  UN  data 
set  can  be  used  as  a  validation  tool  to  assess  the  value  of  USG  data  in  evaluating 
investments.  The  dashed  line  shows  the  approximate  trajectory  that  USG  data  should 
follow  before  it  can  be  used  to  assess  the  perfonnance  of  Department  of  Defense  (DOD) 
investments. 
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Figure  A-3.  Correcting  USG  Data 


This  format  also  allows  analysts  to  examine  the  rates  of  seizures  or  trafficking  of 
various  quantities.  Figure  A-3  illustrates  that  high-risk  events  (involving  larger 
quantities)  occur  at  a  slower  rate  while  low  risk  events  (involving  smaller  quantities) 
occur  at  a  much  faster  rate.  This  is  consistent  with  the  principle  of  equal  risks  for  equal 
returns.  Flowever,  this  is  only  observable  from  the  UN  data,  providing  further  evidence 
that  USG  data  under  samples  small,  low-risk  events. 


Constructing  Surrogate  Data  from  Well  Behaved  Power  Law  Data 

Surrogate  price  data  can  be  extracted  from  the  UN’s  seizure  data  because  it 
appropriately  samples  all  seizure  sizes.  USG  data  does  not  include  the  majority  of  small 
seizure  events,  thus  invalidating  this  technique.  Ultimately,  surrogate  price  data  provides 
the  relative  changes  between  two  values,  therefore  a  scaling  factor  needs  to  be  applied  to 
anchor  it  to  a  realistic  price.  New  analytic  methods  can  be  applied  to  well  behaved  data 
that  sufficiently  sample  the  entire  range  of  seizure  quantities.  IDA  has  tested  and 
validated  this  process  with  USG  cocaine  data.  However,  when  this  method  was  applied  to 
the  Southwest  Asia  (SWA)  data  (from  both  the  UN  and  USG),  only  the  UN  data  yielded 
accurate  results  that  could  be  used  as  surrogate  data. 

Preliminary  technical  analysis  of  UN  data  for  SWA  opium  seizures  supports  the 
finding  that  surrogate  price-like  data  are  approximate  representations  of  real-world  price 
data.  Figure  A-3  demonstrates  why  only  UN  surrogate  data  for  estimating  SWA  opium 
price,  extracted  from  seizure  data,  basically  correlates  with  real  Afghan  opium  prices, 
(regional  and  Afghan  prices  would  be  expected  to  move  in  tandem  over  time.)  The 
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surrogate  data  could  also  be  validated  by  comparing  them  to  other  characteristics  of  the 
drug  market  (for  example,  earlier  2000-2003  regional  price  data  reported  by  the  UN  also 
verified  the  surrogate  process)  to  see  that  they  confonn  to  the  expected  power  law 
distributions  of  drug  data. 

Difficulties  in  Representing  Sequential  Monthly  Small  Samples 

In  order  to  create  and  test  statistical  techniques  that  will  assist  analysts  in  extracting 
valuable  infonnation  from  these  data  sets,  one  must  first  understand  the  nature  of  this 
data.  Although  this  data  almost  universally  exhibits  features  consistent  with  power  laws 
(similar  to  heroin  and  opium  data),  the  sampling  methods  used  may  have  significant 
implications  when  calculating  estimates.  For  example,  although  monthly  averages  are 
commonly  used,  they  are  highly  variable,  which  makes  it  difficult  to  transform  the  data 
into  useful  information. 

Using  the  Median  Over  the  Mean11 

Conventional  analyses  of  average  monthly  seizures  of  illicit  substances  by  law 
enforcement  (LE)  agencies  generally  consider  the  total  weight  of  the  seizures  over  a  fixed 
period  of  time.  Variations  over  different  times  were  assumed  to  result  from  changes  in 
production,  trafficking  rate,  and  market  conditions.  Analysis  of  seizure  data  shows  that 
the  sum  of  seized  quantities  over  a  sampling  period  is  a  metric  that  is  a  poorly  converging 
indicator  of  the  illicit  drug  economy.  The  use  of  the  median  significantly  decreases  the 
variation  and  improves  estimates  of  relative  changes  in  seizure  rates.  The  median  is  a 
better  converging  metric. 

A  very  large  number  of  seizures  (perhaps  as  many  as  half)  may  be  required  to 
improve  estimates  to  within  10%  of  actual  values  or,  in  some  cases,  they  may  remain 
non-convergent.  No  accurate  estimate  may  be  possible  in  some  cases  even  with  large 
quantities  of  seizures  within  the  evaluation  period.  In  many  irregular  war  and  trafficking 
situations,  the  database  is  highly  unlikely  to  contain  more  than  a  small  fraction  of  the  true 
data.  If  so,  the  techniques  presented  here  may  be  the  only  way  to  make  a  useful  estimate. 

Many  of  these  analyses  are  non-intuitive  because  most  common  analyses  use  a  finite 
sample  of  a  larger  population  derived  from  a  sub-period  of  time  (monthly  average)  that 
assumes  that  the  sampled  average  and  quantity  describe  some  (known  or  reasonably 
inferred)  portion  of  the  true  trafficked  quantities.  A  quickly  converging  metric  of  the 


Annex  A,  B,  and  C  of  R.  Anthony  and  A.  Fries,  “Empirical  Modelling  of  Narcotics  Trafficking  from 
Farm  Gate  to  Street,”  in  UNODC  Bulletin  on  Narcotics:  Illicit  Drug  Markets  LVI,  nos.  1  and  2,  (2004) 
show  different  methods  of  analyzing  and  obtaining  valid  market  information  when  the  distributions  are 
as  asymmetric  and  heavy  tailed  as  these.  The  construction  of  a  median  index  by  using  randomized 
percentiles  appears  to  be  superior  to  most  other  techniques. 
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illicit  drug  sample  economy  can  be  shown  to  consist  of  the  total  count  of  seizures  within 
the  same  fixed  time  intervals.  ~  Monte  Carlo  simulations  were  conducted  to 
demonstrate  that  this  metric  achieves  significant  precision  within  hundreds  of  samples 
(versus  thousands).  The  simulation  uses  a  known  distribution  to  calibrate  the  analyses, 
but  uses  estimates  from  the  randomly  extracted  data  to  show  the  approximate  estimation 
error  from  small  samples  of  the  distribution.  Small  samples  create  additional  analysis 
problems  by  overestimating  the  true  values  of  these  asymmetric  distributions. 

In  order  to  more  easily  visualize  and  understand  the  underlying,  bounded, 
distribution,  a  Monte  Carlo  simulation  was  applied  to  test  different  sampled  percentages. 
Random  draws  consisting  of  2,500  samples  each  were  summed  to  form  a  plot  that  shows 
the  distribution  as  nearly  normal  in  shape  with  a  distribution  mode  near  10,000. 
Subsequent  simulations  were  repeated  with  different  sample  sizes  (e.g.,  250  for  10%  or 
25  for  1%)  to  assess  the  effect  of  small  samples.  After  the  different  percentages  were 
obtained,  each  sample  was  scaled  to  a  size  comparable  to  the  original  distribution  in  order 
to  more  clearly  show  the  sampling  effect. 

Figure  A-4  shows  how  the  average  (the  red  arrow,  computed  from  the  raw  sample 
of  the  Monte  Carlo  simulation)  steadily  increases  as  the  sample  size  decreases.  In  this 
series  of  charts,  the  blue  line  is  the  median,  i.e.,  where  there  are  an  equal  number  of 
events  on  either  side  of  the  middle  value  (fiftieth  percentile).  The  red  arrow  is  the  sample 
average  and  the  peak  is  the  mode.14  Comparing  the  top  and  bottom  simulations 
(representing  a  100%  and  1%  sample  size,  respectively)  shows  how  divergent  the  two 
means  are  and,  therefore,  why  the  use  of  averages  is  not  a  good  estimator  of  the  system 
(all  are  the  same  scale). 


In  Barry  D.  Crane,  A.  Rex  Rivolo,  and  Gary  C.  Comfort,  An  Empirical  Examination  of  Counterdrug 
Interdiction  Program  Effectiveness,  IDA  Paper  P-3219  (Alexandria,  VA:  Institute  for  Defense 
Analyses,  1997),  the  detailed  construction  of  a  median  index  is  described  to  maximize  the  extracted 
information  content  from  difficult  data. 

A  Monte  Carlo  simulation  is  a  type  of  computational  simulation  that  relies  on  repeated  random 
sampling  to  compute  results. 

Crane,  Rivolo,  and  Comfort,  An  Empirical  Examination.  This  paper  was  the  first  to  document  how  in 
unknown  (especially  power  law-like)  distributions  only  the  medians  might  converge.  Testing  these 
analyses  over  ten  years  confirms  the  practical  use  of  these  techniques. 
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A  typical  monthly  seizure 
average  of  SWA  data  has 
a  sample  size  of  less  than 
1% 


Figure  A-4.  Monte  Carlo  Simulations  of  Progressively  Smaller  Sample  Sizes  and  the 
Corresponding  Overestimation  of  Averages 
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Figure  A- 5  uses  actual  operational  data  from  attacks  on  an  Iraqi  airbase  to  show  that 
IDA  techniques  have  a  much  broader  application  than  drug  data  (a  more  detailed 
explanation  is  presented  in  Appendix  C).  It  shows  that  averages  (100  attack  events — blue 
dots)  appear  not  to  converge  when  the  number  of  attacks  decline  to  small  values  (where 
the  largest  differences  occur)  of  the  total  distribution  and  the  intervals  between  attacks 
grow  larger  from  a  successful  defense.  When  the  number  of  attacks  declined  by 
approximately  75%,  the  average  difference  grew  as  the  sampling  became  smaller;  and  it 
is  obvious  that  the  random  variation  of  average  attack  intervals  is  unpredictable  from 
negative  excursions  of  -30%  to  positive  excursions  greater  than  +150%. 
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Figure  A-5.  Random  Average  Fluctuations  in  Attack  Intervals  over  Five  Years  with 
Increasing  Differences  as  illustrated  by  the  Red  Arrows 


This  example  shows  how  the  use  of  averages  is  unpredictable  and  does  not  help 
Commanders  assess  when  their  interventions  are  successful.  The  median  is  a  much  more 
precise  and  practical  estimate  to  help  determine  when  tactics  are  effective.  As  in  Iraq,  this 
concept  can  be  applied  to  sparse  SWA  data  to  detennine  the  success  of  very  large  DOD 
investments  in  security. 


A-10 


Impact  of  Threshold  Effects  in  Evaluating  Performance 

Besides  determining  the  analytical  power  law  distributions,  it  is  also  necessary  to 
detennine  thresholds  for  effective  actions.  The  thresholds  have  been  observed  repeatedly 
with  respect  to  insurgents  and  traffickers.  Figure  A-6  shows  typical  characteristics  of 
traffickers  or  insurgents  learning  from  insufficient  interventions.  When  critical  thresholds 
are  reached,  insurgent  and  trafficking  activities  decline  abruptly. 15  Threshold 
detennination  from  real  data  requires  estimation  of  unknown  parameters.  Although  this 
process  is  demonstrated  in  the  analysis  of  the  battle  for  Iraq,  the  study  uses  various  well 
known  drug  data  and  other  activities  to  illustrate  the  underlying  principles.16  Since 
irregular  adversaries  leam  from  USG  operations,  they  can  improve  their  methods — in 
economics  this  is  known  as  a  “progress  curve.”  Deterrence  is  vitally  important  because  it 
greatly  improves  the  efficiency  of  intervention  operations.  Panama  Express  and  Joint 
Interagency  Task  Force  South  operations  achieved  a  four-fold  improvement  in 
performance  with  no  additional  resources  or  costs.  Figure  A-6  provides  a  simple 
illustration  of  progress  and  deterrence. 


Drug  trafficking  declines  were  observed  from  the  above  threshold  operations:  in  air  trafficking  in  Peru 
and  the  Caribbean  Sea,  the  decline  of  go-fast  trafficking  through  the  Caribbean,  the  defense  of  Puerto 
Rico,  and  finally,  the  70%  decline  in  cocaine  flow  through  Mexico  are  all  examples.  In  combat 
operations,  detailed  tactical  deterrence  was  observed  at  checkpoints,  in  many  different  specific  counter 
tactics,  in  the  defense  at  Joint  Base  Balad,  and  in  overall  insurgent  activities  in  Iraq. 

Joseph  A.  Milner  and  Barry  D.  Crane,  Integrated  Defense:  Lessons  Learned  from  Joint  Base  Balad, 
IDA  Document  NS  D-4372  (Alexandria,  VA:  Institute  for  Defense  Analyses,  August  2011). 
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Figure  A-6.  Typical  Observed  Behavior  of  the  Onset  of  Deterrence  from  “Enemy/Trafficker” 

Operations 


The  implication  of  this  research  is  that  counter-trafficking  operations  should  be 
conducted  with  sufficient  enforcement  actions  to  modify  insurgents’  and  traffickers’ 
behaviors  by  exceeding  the  risk  return  curves,  i.e.,  to  deter  them.  This  effect  is 
determined  by  thresholds:  below  the  threshold  the  adversary  leams  from  enforcement 
operations,  above  the  threshold  some  begin  to  be  deterred. 

The  following  discussion  of  deterrence  is  paraphrased  from  original  IDA  research 
that  is  still  valid.  When  irregular  forces  and  traffickers  are  deterred,  they  are  repeatedly 
observed  to  respond  inversely  to  an  intervention  (power  law  exponent  -1)  making 
analyses  difficult.  The  functions  described  here  were  derived  directly  from  observations 
alone  and  not  any  network  theory,  game  theory,  or  other  kind  of  underlying  process. 

IDA  has  perfonned  extensive  research  since  the  mid-1990s  on  deterrence  and  its 
impact  on  operational  effectiveness. 19  This  includes  the  Rockwell  interviews  of  captured 


Classified  risk-return  curves  for  enemy  combatants  define  their  ranges  of  observed  acceptable  risks. 
These  analyses  techniques  are  general  in  nature,  widely  known,  and  apply  to  a  large  set  of  irregular 
conflicts.  See  R.  Anthony,  “A  Calibrated  Model  of  the  Psychology  of  Deterrence,”  UNODC  Bulletin 
on  Narcotics'.  Illicit  Drug  Markets  LVI,  nos.  1  and  2  (2004). 

Robert  W.  Anthony,  Barry  D.  Crane,  and  Stephen  F.  Hanson,  Deterrence  Effects  from  Peru ’s  Force- 
down/Shoot-down  Policy ,  IDA  Paper  P-3472  (Alexandria,  VA:  Institute  for  Defense  Analyses,  April 
2000),  20-32. 

Ibid.,  Appendix  A.  Also  see  R.  Anthony,  “A  Calibrated  Model  of  the  Psychology  of  Deterrence,” 
UNODC  Bulletin  on  Narcotics:  Illicit  Drug  Markets  LVI,  nos.  1  and  2  (2004). 
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smugglers,  which  were  remarkably  useful  for  determining  the  underlying  mathematical 
distribution  of  the  “willingness  to  traffic”  function.  A  single  power  law  exponent  was 
found  to  represent  all  seven  sets  of  conditions  (from  the  three  categories:  lethal  force, 
arrests  with  conviction,  and  material  loss)  for  evaluating  deterrence: 

1 .  Material  Loss  to  Capture 

2.  Capture  to  Prison 

3.  Prison  to  Loss  of  Life 

4.  Self  Caught  (individuals  view  of  risk) 

5.  Associate  Caught  (effect  of  an  individual's  colleague  being  caught) 

6.  Self  Imprisoned 

7.  Associate  Imprisoned 

Above  the  threshold,  where  W  is  the  willingness  to  smuggle  or  commit  the  act  from 
the  probability  of  interdiction  (PO  and  the  threshold  denoted  by  Pmin  and  the  following 
condition  describes  this: 


v -1.03  ±0.07 


for  />  >Pnmi 
for  Pj  < 


The  values  Pmin  describe  the  thresholds  for  three  approximate  sets  of  risk  conditions: 
Kinetic  operations:  2%  to  5%,  nominally  3% 

Arrest  operations:  5%  to  13%,  nominally  8% 

20 

Property  seizure/loss  operations:  13%  to  30%,  nominally  24% 

The  following  equation  depicts  the  willingness  to  smuggle.  One  interesting 
observation  is  that  there  are  similar  thresholds  observed  in  the  analyses  of  combat  data 
(see  Appendix  C).  A  second  observation  is  the  fraction  that  is  not  deterred,  as  shown  in 

1  029  21 

the  term  Pm]n  as  undeterred"  and  dependent  on  the  intervention  risks. 


Ibid.,  Appendix  A. 

There  are  cases  where  some  fraction  of  criminals,  even  if  they  know  they  are  going  to  be  caught, 
commit  the  crime,  especially  when  their  families  are  threatened  or  some  other  factor  outweighs  the 
penalty  for  getting  caught. 
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Figure  A-7.  Willingness  to  Smuggle  [W(PI)] 

From  the  willingness  function,  the  deterrence  function,  i.e.,  the  probability  of 
thwarting  (=  deterrence  +  interdiction)  can  be  easily  derived  as: 


Pt=l-(l-PI). 

=  1-(1  -Pi) 


P 

nun 


v— 1.03±0.07 


for  Pj  >  Pm 
for  Pj  <  Pm 


The  threshold  conditions  describe  a  non-linear  break  point  and  the  transition  to  a 
power  law  with  an  exponent  of  -1.  All  deterrence  conditions  that  have  been  observed 
seem  to  obey  the  power  law  and  the  difficult  mathematics  needed  to  analyze  this 


22 


Ibid. 
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situation.  This  condition  reinforces  the  argument  against  using  averages  rather  than 
medians  in  sampling  strategies  to  assess  performance  or  in  attempting  to  quantify  the 
system.  Figure  A-8  shows  the  deterrence  function  as  a  graph: 


Probability  of  Thwarting  Smuggling 
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Figure  A-8.  Deterrence  Graphical  Formulation  (Power  Law  Exponent  =  -1) 


The  many  conditions  shown  in  Figure  A-8  essentially  breakdown  into  loss  of  life, 
arrest  with  conviction,  and  material  loss.  The  ranges  of  these  effects  are  also  shown  in 
Figure  A-8.  In  summary,  this  view,  developed  in  the  1990s,  was  repeatedly  tested  and  is 
useful  in  planning  operations,  efficiently  using  resources,  and  assessing  outcomes. 

Figure  A-8  illustrates  the  probability  of  thwarting  drug  smuggling  (otherwise  known 
as  deterrence,  measured  on  the  y  axis)  that  can  be  expected  for  any  given  level  of 
interdiction  resources  (x  axis).  The  lower  the  probability  of  interdiction,  the  more  severe 
the  action  that  must  be  taken  by  LE  to  achieve  deterrence.  In  other  words,  given  a  very 
low  level  of  interdiction  resources,  a  government  may  need  to  use  lethal  force  to  prevent 
the  movement  of  drugs.  Or,  the  government  can  commit  a  great  deal  more  assets  and 
make  many  more  arrests  to  have  the  same  deterrent  effect.  From  the  trafficker’s 


23 


Ibid. 
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perspective,  the  use  of  lethal  force  (personal  death  and  associate  death)  has  the  greatest 
deterrent  effect,  followed  by  risk  of  arrest  (personal  and  associate),  and  then  loss  of  drugs 
and  equipment. 

The  text  below  quantifies  this  deterrence  effect  mathematically  for  deterrence 
efforts  in  Peru.24 

•  Lethal  Force  (Pmin  <  0.02):  With  the  threat  of  lethal  force,  traffickers  begin  to 
quit  challenging  the  interdictors  when  Pmin  reaches  about  2%.  Much  below  this 
threshold,  however,  traffickers  are  willing  to  accept  the  risks  as  a  cost  of  doing 
business. 

•  Personal  Imprisonment  (0.02  <  Pmin  <  0.05):  If  experienced  traffickers 
anticipate  a  severe  sentence  whenever  they  are  captured,  they  will  begin  to  be 
significantly  deterred  in  this  range  of  interdiction  probabilities. 

•  Capture  and  Imprisonment  of  an  Associate  (0.05  <  Pmin  <  0.13):  Those  who 
have  not  experienced  prison  life  may  be  more  difficult  to  deter  and  require 
thresholds  in  the  range  from  5  to  13%. 

•  Loss  of  Boat  or  Aircraft  (0.13  <  Pm;n  <  0.3):  This  zone  of  interdiction  threat 
includes  loss  of  the  drugs  as  well. 

•  Loss  of  Drugs  (0.3  <  Pm;n  <  1):  Interviews  with  inmates  and  observed 
avoidance  behavior  when  threatened  with  interdiction  indicate  another  zone  of 
loss  of  drugs. 

Many  operations  were  conducted  utilizing  tactics  and  strategies  developed  from  this 
deterrence  concept  and  the  underlying  power  law.  Median  estimates,  versus  average 
estimates,  routinely  aided  in  understanding  the  degree  of  success  attained  by  individual 
counter-cocaine  operations  and  trends  exhibited  across  a  series  of  operations.  Threshold 
conditions  (Pmin)  are  shown  in  Figure  A-7  and  Figure  A-8.  Deterrence  effects  have  also 
been  used  to  affect  consumption. 

Figure  A-9  illustrates  the  empirical  relationship  between  price  and  user  rates.  This 
analysis  was  mandated  by  several  Presidential  Directives  that  sought  to  control  the 
cocaine  epidemic  by  driving  up  cocaine  prices  to  decrease  consumption.  The  eight  red 
spikes  indicating  price  increases  were  a  result  of  repeatedly-tested  deterrence  actions  that 
reduced  supply.  Drops  in  positive  cocaine  testing  rates  confirmed  this  reduction  in 
supply.  More  importantly,  this  figure  illustrates  that  consistent  and  repeated  supply 
attacks  significantly  altered  the  recovery  of  user  demand  as  measured  by  positive  test 
rates.'  One  of  the  most  critical  observations  is  the  elasticity  and  relationships  between 


Anthony,  Crane,  and  Hanson,  Deterrence  Effects  from  Peru's  Force-down/Shoot-down  Policy,  20-32. 
Crane,  Rivolo,  and  Comfort,  An  Empirical  Examination. 
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supply  (price  changes)  and  demand  (positive  test  rates)  power  laws.  The  user  testing  rates 
drop  when  prices  rise,  but  when  prices  recover,  user  testing  rates  recover  initially  at  a 
slow  rate,  and  after  repeated  attacks  hardly  recover  at  all.  This  critical  observation  is  the 
basis  for  estimating  the  time  necessary  for  allied  forces  to  conduct  effective  major 
operations  to  gain  the  initiative  against  cocaine  trafficking — euphemistically  called 
“getting  inside  the  enemy’s  response  loop.” 
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Note:  pgm=pure  gram,  GWF=general  workforce,  and  Oz=ounce.  Numbers  indicate  the  peak  of  identified 
operation. 
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Figure  A-9.  The  Relationship  Between  Cocaine  Price  and  Positive  Test  Rates  (Demand) 


This  deterrence  fonnulation  clearly  explained  how  power  law  type  data  can  be 
transformed  into  valuable  infonnation  and  how  repeated  analytical  applications  lead  to 
strategic,  theater-wide  effects. 


Test  rates  data  is  provided  by  the  Quest  Corporation  which  tests  the  general  workforce  employees 
(600,000/year).  Price  data  is  from  the  System  to  Retrieve  Drug  Evidence  (STRIDE)  database  which 
constructs  a  price  index. 
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Appendix  B 

Analysis  of  the  Cocaine  Market 


The  Institute  for  Defense  Analyses’  (IDA)  initial  research  on  power  laws  pertained 
to  the  cocaine  market  in  the  Western  Hemisphere.  This  appendix  contains  examples  of 
how  power  laws  applied  to  most  aspects  of  the  cocaine  system,  such  as  assuming  market 
risk  and  consumption.  Having  demonstrated  that  the  cocaine  system  exhibits  behavior 
consistent  with  power  laws,  it  is  logical  to  assume  that  the  opium  and  heroin  market  data 
and  consumption  will  also  follow  the  same  distribution.  IDA’s  analysis  suggests  that 
these  functional  distributions  characterize  not  only  the  cocaine  market,  but  the  heroin, 
marijuana,  methamphetamine,  and  methylenedioxymethamphetamine  (MDMA)  markets 
as  well.  Because  of  the  pervasive  and  ubiquitous  nature  of  drug  data  distributions,  IDA’s 
research  findings  from  the  Western  Hemisphere  can  be  applied  to  Southwest  Asia  (SWA) 
heroin  and  opium  markets.  IDA  Document  D-4324,  Phase  II  Findings:  Leveraging 
Lessons  Learned  to  Inform  Southwest  Asia  Counter-Drug  Efforts 1  provides  further 
evidence  that  SWA  heroin  and  opium  will  also  exhibit  the  power  laws  presented  in  this 
appendix. 

In  the  cocaine  market,  power  laws  effectively  characterize  the  behaviors  of  users, 
buyers,  and  sellers  of  drugs  who  assume  market  risk.  More  criminals  sell  smaller 
amounts  since  it  is  much  less  risky  than  selling  larger  quantities.  IDA  asserts  that  similar 
conditions  likely  exist  for  SWA  heroin  and  opium  markets.  Preliminary  tests  of  SWA 
data  confirm  this  assertion.  Figure  B-l  shows  price  per  pure  gram  versus  quantity 
purchased  in  then- year  dollars  in  the  1990s. 


Barry  D.  Crane,  Ashley-Louise  N.  Bybee,  Andrew  Cseko,  Andrea  S.  Pongo,  and  Zachary  S.  Rabold, 
Phase  II  Findings:  Leveraging  Lessons  Learned  to  Inform  Southwest  Asia  Counter-Drug  Efforts,  IDA 
Document  D-4324  (Alexandria,  VA:  Institute  for  Defense  Analyses,  May  2011). 
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Source;  System  To  Retrieve  Information  from  Drug  Evidence  (STRIDE)  database. 

Figure  B-1.  Cocaine  Purchase  Quantity  Relationships2 

Figure  B-2  illustrates  that  this  same  characteristic  exists  for  all  drugs,  including 
cocaine,  heroin,  methamphetamine,  marijuana,  and  MDMA.  In  other  words,  they  all  have 
a  similar  price  discount  behavior.  This  is  consistent  with  the  underlying  organizational 
structure  of  criminal  networks  selling  illegal  substances. 


See  R.  Anthony  and  A.  Fries,  “Empirical  Modelling  of  Narcotics  trafficking  from  Farm  Gate  to 
Street,”  UNODC  Bulletin  on  Narcotics:  Illicit  Drug  Markets  LVI,  nos.  1  and  2  (2004):  1 1,  Figure  III, 
for  a  more  complete  description  of  these  data. 

3  Ibid. 
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Source:  System  To  Retrieve  Information  from  Drug  Evidence  (STRIDE)  database. 

Figure  B-2.  All  U.S.  Drugs  and  Purchase  Quantity  Relationships4 

The  data  presented  in  Figure  B-2  supports  the  assertion  that  these  drugs  have  the 
same  price-amount  percentage  markup  because  the  underlying  organizational  structure 
exhibits  the  same  risk  mitigation  tendency.  Similar  organizational  structures  exist  in 
SWA  and  subsequent  analysis  of  SWA  data  can  be  expected  to  reveal  comparable  power 
law  depictions.  Since  many  of  the  organizations  do  not  know  or  even  communicate  with 
each  other,  this  observed  behavior  is  consistent  with  a  fundamental  underlying  principle 
of  risk-taking:  equal  risks  for  equal  returns. 

The  power  law  characterizing  the  entire  cocaine  market  is  a  powerful  insight  and 
applies  to  the  entire  trafficking  system,  from  cultivation  in  the  source  zone,  to  laboratory 
production  of  cocaine  hydrochloride  (HC1),  to  final  retail  distribution  in  user  markets,  as 
presented  in  Figure  B-3. 


Ibid.,  12,  Figure  IV  for  a  more  complete  description  of  these  data. 
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Figure  B-3.  Price  Quantity  Trends  for  Consolidation  and  Distribution5 

This  vertical  trafficking  system  typically  imposes  a  price  markup  of  about  2.5  times 
per  stage  as  the  different  organizations  parse  risk  for  each  stage  of  trafficking.  Ultimately 
all  elements  of  this  vertical  trafficking  system — from  coca  farmers  to  the  casual  user — 
follow  this  power  law  and  thus  the  euphemism,  consistent  from  “leaf  to  street,”  applies. 

Using  this  power  law  structure,  analysts  subsequently  tested  United  Nations  global 
cocaine  price  data  and  found  several  anomalous  features  in  the  data.  Recognizing  that 
power  laws  are  observed  to  apply  in  all  aspects  of  the  drug  trafficking  system,  this 
prompted  analysts  to  question  unexplained  fluctuations  in  the  data,  such  as  where 
declining  prices  (indicating  a  substantial  supply)  emerged  in  unexpected  places.  The 
power  law  representation  of  markups,  shown  in  Figure  B-3,  held  for  global  markets,  but 
if  the  mark  ups  suddenly  change,  then  these  prices  show  hidden  or  covert  trafficking  or 
bad  data.  Multiple  checks  of  other  related  data,  such  as  consumption,  confirm  changes. 
Figure  B-4  shows  how  these  price  relationships,  in  a  steady-state  cocaine  market 
observed  in  2007,  suddenly  change  when  they  did  not  follow  the  same  mark  up  structure. 
Other  data  confirm  these  dramatic  changes.  Deviations  from  the  steady-state  conditions 
reveal  previously  unknown  patterns  of  trafficking — particularly  the  declines  in  prices  in 
the  region  around  Syria. 


Ibid.,  16,  Figure  V. 
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Data  Source:  UNODC  Delta  Database  provided  by  the  research  section  in  2008 

Figure  B-4.  The  UN  Reported  Global  Price  Network  with  Highlighted  Changes 

This  anomaly  allowed  analysts  to  uncover  previously  unknown  trafficking  patterns. 
This  information  also  aided  in  uncovering  and  targeting  these  new  supply  routes  and  has 
resulted  in  arrests,  extraditions,  and  trials. 

Drug  use  (demand)  is  also  characterized  by  a  power  law,  since  heavy  drug  users 
take  more  risks  than  light  drug  users.  The  relative  consumption  of  heavy  users  (about 
20%  of  total  users  and  monthly  or  perhaps  weekly  use)  is  about  80%  of  the  total  quantity 
of  drugs,  while  light  users  (about  80%  of  total  users  who  are  perhaps  annual  or  just 
lifetime  users)  consume  about  20%  of  the  total  quantity  of  drugs.  This  is  a  classic 
example  of  a  power  law  and  is  known  as  the  “80-20  rule”  in  the  statistical  literature. 
Figure  B-5  illustrates  this  consumer  behavior. 
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Specific  consumption,  X  (g/yr) 

Fitted  curve  — o —  Lifetime  use  — * —  Annual  use 

— is —  Monthly  use  Integration  limits 

U  =  Number  of  users  in  millions;  Q  =  Total  consumption  in  metric  tons 
Source:  United  States  National  Household  Survey  on  Drug  Abuse,  1990-1994. 
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Figure  B-5.  Power  Law  Describing  Differential  Rate  of  Cocaine  Use 


Ibid.,  5,  Figure  I. 
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Appendix  C 

Analysis  of  Combat  Data 


Analyses  of  the  attacks  on  the  Joint  Iraqi  Airbase  Balad  provide  a  simple 
demonstration  of  the  complexities  of  power  laws  and  of  the  fact  that  the  monthly  average 
is  not  a  good  indicator  of  change  for  assessing  the  success  of  deterrence  operations.1 
Using  drug  data  analyses  techniques  and  understanding  the  underlying  organization  of 
the  attackers  on  Joint  Base  Balad  was  helpful  for  analyzing  when  the  attacks  began  to  be 
deterred.  What  was  learned  regarding  drug  criminal  behavior  could  be  used  to  understand 
how  well  the  base  was  defended. 

A  power  law  also  applies  to  anti-Iraqi  insurgent  and  coalition  forces’  willingness  to 
incur  casualties  in  combat  and  indirect  attacks  on  installations.  This  application  may 
assist  analysts  in  identifying  the  likelihood  of  a  future  event  occurring,  but  cannot  predict 
the  exact  timing  of  an  event. 

These  observations  and  hypotheses  have  been  used  to  uncover  more  detailed 
behaviors  of  insurgent  combat  groups  in  both  Iraq  and  Afghanistan.  Analyses  of  the 
attack  on  Joint  Base  Balad  in  Iraq  demonstrate  all  of  the  power  law  analysis 
characteristics.  Figure  C-l  shows  the  median  number  of  monthly  attacks  (top  diagram) 
and  the  median  intervals  between  attacks  (bottom  diagram).  It  also  shows  the  large  errors 
that  result  from  using  monthly  averages,  and  why  averages  are  problematic  in  assessing 
the  success  of  operations. 


For  more  detail  see  Barry  D.  Crane,  A.  Rex  Rivolo,  and  Gary  C.  Comfort,  An  Empirical  Examination 
of  Counterdrug  Interdiction  Program  Effectiveness,  IDA  Paper  P-3219  (Alexandria,  VA:  Institute  for 
Defense  Analyses,  1997).  Other  techniques  for  examining  these  power  law  situations  are  described  in 
UNODC  Bulletin  on  Narcotics:  Illicit  Drugs,  2004,  Chapters  1  and  2. 

Data  are  available  in  classified  combat  reporting. 

Joseph  A.  Milner,  Barry  D.  Crane,  Integrated  Defense:  Lessons  Learned  from  Joint  Base  Balad,  IDA 
Document  NS  D-4372  (Alexandria,  VA:  Institute  for  Defense  Analyses,  August  2011). 
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Note:  Each  blue  dot  is  an  attack  interval  while  the  line  is  a  median  monthly  average 

Figure  C-1.  Median  Number  of  Monthly  Attacks  (top)  and  the  Median  Intervals  Between 

Attacks  (bottom)4 


Figure  C-2  shows  that  the  time  intervals  (days)  between  attacks  also  follow  an 
approximate  power  law  \Number=450*\Days]~l  °87]  with  an  exponent  of  -1.087. 
Interestingly,  this  exponent  is  almost  identical  to  the  one  derived  from  an  analysis  of  the 
rate  of  cocaine  trafficking,  reflecting  similar  behaviors  of  traffickers  with  regard  to  risk¬ 
taking. 


4  Ibid.,  39. 
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Note:  Blue  dots  are  the  number  of  attacks  per  attack  interval  in  days  (1 , 2,  3,  4,  20,  30,  and  40) 

Figure  C-2.  Relationship  Between  the  Number  and  Rate  of  Attacks  (since  2006)5 

When  the  power  law  shape  exponent  is  near  - 1 ,  as  it  is  in  this  case,  averages  are  a 
very  poor  measure  of  effectiveness.  Figure  C-3  demonstrates  the  ineffectiveness  of  using 
averages  to  describe  the  interval  data  (described  earlier,  but  added  for  clarification  here). 
The  average  shows  random  fluctuations  of  growing  magnitude  making  it  difficult  to 
determine  any  effectiveness  at  all.6  The  average  percent  error  is  a  result  of  comparing  the 
average  to  a  stable  indicator,  such  as  the  median.  In  nonconvergent  systems,  using 
different  starting  points  reveals  a  completely  different  set  of  error  results  with  no 
correlation  to  previous  values. 


Ibid.,  41. 

COL  Joseph  Milner,  as  senior  security  group  commander,  confirmed  this  difficulty  in  2007.  His 
analyses  confirmed  the  operational  assessment  of  the  commander.  The  typical  security  group  data 
analyses  made  it  very  difficult  to  observe  success.  See  NSD-4372  waiting  publishing  at  Air  University. 
Used  with  permission  of  COL  Milner. 
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7  Ibid. 
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Appendix  F 
Abbreviations 


AOR 

CN&GT 

CCDB 

CCDB-E 

CENTCOM 

CIA 

CIDNE 

CN 

CNP 

CNPA 

COCOM 

COIN 

CONUS 

D&M 

DASD 

DEA 

DIA 

DOD 

FOUO 

g/yr 

GWF 

HC1 

IACM 

IAG 

IDA 

ISAF 

kg 

LE 

MDMA 

mt 

Oz 

PAR 

PGM 

SIGACT 

SIMCI 

STRIDE 

SWA 


Area  of  Responsibility 

Counternarcotics  and  Global  Threat 

Consolidated  Counterdrug  Database 

Consolidated  Counterdrug  Database-East 

United  States  Central  Command 

Central  Intelligence  Agency 

Combined  Infonnation  Data  Network  Exchange 

Counternarcotics 

Colombian  National  Police 

Countemarcotics  Police  of  Afghanistan 

Combatant  Command 

C  ounterinsurgency 

Continental  United  States 

Detection  and  Monitoring 

Deputy  Assistant  Secretary  of  Defense 

Drug  Enforcement  Administration 

Defense  Intelligence  Agency 

Department  of  Defense 

For  Official  Use  Only 

Grams  per  year 

General  Workforce 

Cocaine  Hydrochloride 

Interagency  Assessment  of  Cocaine  Movement 

Interagency  Group 

Institute  for  Defense  Analyses 

International  Security  Assistance  Force 

kilogram 

Law  Enforcement 

methylenedioxymethamphetamine 

Metric  Tons 

Ounce 

Performance  Assessment  Review 
Pure  Gram 
Significant  Activities 

Sistema  Integrado  de  Monitoreo  de  Cultivos  Ilicitos 
(Integrated  Crops  Monitoring  System) 

System  To  Retrieve  Information  from  Drug  Evidence 
Southwest  Asia 
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TZ 

Transit  Zone 

UN 

United  Nations 

UNODC 

United  Nations  Office  on  Drugs  and  Crime 

USG 

United  States  Government 

WH 

Western  Hemisphere 
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